2024-09-18T00:52:00.609 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T00:52:00.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T00:52:00.704 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909691' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBISZwA441xiMZLm497ZwZmlSvie/fcJ2cbYVRqn9xFwMiYDEIoXm5JaPJ92n/si9xlyr7i0+vFjiam3tDs4pvNY= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBkQo1omx3/jkcfnR/ZRolbFMe+tksQMdhFSiGOehaiffUP476vaXBc9E8oyR434mnC6T9iF/17QImCUj6fB57k= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMQFH3eaEC9zyNXB5ioUQB3PxZOiB+5M328saKGTTH0wW6rCaPvqCJnpf4v8b0lcyeObJBWJqGmtuioz+TQ9dTM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T00:52:00.704 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-18T00:52:00.705 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-18T00:52:00.705 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T00:52:00.707 INFO:teuthology.task.internal:Checking packages... 2024-09-18T00:52:00.727 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-18T00:52:00.727 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T00:52:00.727 INFO:teuthology.packaging:ref: None 2024-09-18T00:52:00.727 INFO:teuthology.packaging:tag: None 2024-09-18T00:52:00.727 INFO:teuthology.packaging:branch: squid 2024-09-18T00:52:00.727 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:52:00.727 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-18T00:52:00.964 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-0 2024-09-18T00:52:00.965 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T00:52:00.972 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T00:52:00.973 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T00:52:00.988 INFO:teuthology.task.internal:Saving configuration 2024-09-18T00:52:01.001 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T00:52:01.007 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T00:52:01.027 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 00:47:03.561782', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBISZwA441xiMZLm497ZwZmlSvie/fcJ2cbYVRqn9xFwMiYDEIoXm5JaPJ92n/si9xlyr7i0+vFjiam3tDs4pvNY='} 2024-09-18T00:52:01.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 00:47:03.557964', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBkQo1omx3/jkcfnR/ZRolbFMe+tksQMdhFSiGOehaiffUP476vaXBc9E8oyR434mnC6T9iF/17QImCUj6fB57k='} 2024-09-18T00:52:01.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 00:47:03.559990', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMQFH3eaEC9zyNXB5ioUQB3PxZOiB+5M328saKGTTH0wW6rCaPvqCJnpf4v8b0lcyeObJBWJqGmtuioz+TQ9dTM='} 2024-09-18T00:52:01.062 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T00:52:01.069 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-18T00:52:01.069 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-18T00:52:01.069 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['client.0'] 2024-09-18T00:52:01.069 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T00:52:01.142 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f717d8a7ac0>, signals=[15]) 2024-09-18T00:52:01.142 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T00:52:01.151 INFO:teuthology.task.internal:Opening connections... 2024-09-18T00:52:01.151 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T00:52:01.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:52:01.227 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T00:52:01.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:52:01.299 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-18T00:52:01.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:52:01.372 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T00:52:01.379 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-18T00:52:01.397 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-18T00:52:01.398 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-18T00:52:01.456 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-18T00:52:01.456 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-18T00:52:01.456 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-18T00:52:01.456 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-18T00:52:01.456 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-18T00:52:01.456 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T00:52:01.457 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T00:52:01.458 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-18T00:52:01.482 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-09-18T00:52:01.501 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-09-18T00:52:01.501 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-09-18T00:52:01.559 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-09-18T00:52:01.559 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-09-18T00:52:01.559 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-09-18T00:52:01.559 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-09-18T00:52:01.559 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-09-18T00:52:01.559 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-09-18T00:52:01.559 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T00:52:01.560 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-09-18T00:52:01.560 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-09-18T00:52:01.560 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T00:52:01.560 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-09-18T00:52:01.560 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T00:52:01.560 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T00:52:01.560 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T00:52:01.560 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-09-18T00:52:01.582 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-18T00:52:01.601 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-18T00:52:01.601 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T00:52:01.659 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-09-18T00:52:01.660 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T00:52:01.660 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T00:52:01.660 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T00:52:01.660 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-18T00:52:01.683 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T00:52:01.691 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T00:52:01.698 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T00:52:01.698 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:52:01.701 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:52:01.703 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-18T00:52:01.720 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T00:52:01.726 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T00:52:01.726 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:52:01.758 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:52:01.760 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-18T00:52:01.790 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T00:52:01.869 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T00:52:01.884 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T00:52:01.884 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T00:52:01.884 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:52:01.885 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:52:01.885 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T00:52:01.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:52:01.902 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-18T00:52:01.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:52:01.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:52:01.904 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-18T00:52:01.904 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-18T00:52:01.960 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-503.el9.x86_64 2024-09-18T00:52:01.960 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-503.el9.x86_64 2024-09-18T00:52:01.961 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-09-18T00:52:01.962 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-503.el9.x86_64 2024-09-18T00:52:01.962 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-18T00:52:01.962 INFO:teuthology.task.kernel:Running kernel on smithi098: 5.14.0-503.el9.x86_64 2024-09-18T00:52:01.963 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum install -y kernel 2024-09-18T00:52:01.963 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-18T00:52:01.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-18T00:52:04.316 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-18T00:52:04.530 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.3 MB 00:00 2024-09-18T00:52:04.638 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.3 MB 00:00 2024-09-18T00:52:06.697 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-18T00:52:06.833 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-18T00:52:11.034 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-18T00:52:12.318 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-18T00:52:12.465 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 113 kB/s | 19 kB 00:00 2024-09-18T00:52:13.844 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-18T00:52:14.669 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 11 MB/s | 23 MB 00:02 2024-09-18T00:52:15.998 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 11 MB/s | 23 MB 00:02 2024-09-18T00:52:16.938 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 20 MB 00:11 2024-09-18T00:52:20.158 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-18T00:52:21.395 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-18T00:52:22.089 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T00:52:22.090 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T00:52:22.191 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T00:52:22.206 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T00:52:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-18T00:52:22.206 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T00:52:22.206 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T00:52:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T00:52:22.206 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T00:52:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T00:52:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T00:52:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T00:52:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:52:22.207 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T00:52:22.207 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T00:52:22.207 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-18T00:52:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:52:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-18T00:52:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-18T00:52:22.208 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T00:52:22.451 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 303 kB/s | 36 kB 00:00 2024-09-18T00:52:23.228 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-18T00:52:23.268 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T00:52:23.268 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T00:52:23.348 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T00:52:23.358 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T00:52:23.358 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2024-09-18T00:52:23.358 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T00:52:23.358 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T00:52:23.358 INFO:teuthology.orchestra.run.smithi098.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T00:52:23.358 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout:Install 4 Packages 2024-09-18T00:52:23.359 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:52:23.360 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 79 M 2024-09-18T00:52:23.360 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 126 M 2024-09-18T00:52:23.360 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T00:52:23.446 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-18T00:52:23.729 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 22 MB/s | 29 MB 00:01 2024-09-18T00:52:23.738 INFO:teuthology.orchestra.run.smithi098.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 293 kB/s | 36 kB 00:00 2024-09-18T00:52:24.095 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-18T00:52:24.096 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:52:24.096 INFO:teuthology.orchestra.run.smithi031.stdout:Total 42 MB/s | 79 MB 00:01 2024-09-18T00:52:24.811 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T00:52:25.192 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T00:52:25.192 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T00:52:25.722 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-18T00:52:26.184 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T00:52:26.185 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T00:52:26.938 INFO:teuthology.orchestra.run.smithi098.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.7 MB/s | 16 MB 00:03 2024-09-18T00:52:27.567 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 13 MB/s | 23 MB 00:01 2024-09-18T00:52:28.613 INFO:teuthology.orchestra.run.smithi098.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.9 MB/s | 29 MB 00:04 2024-09-18T00:52:28.753 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T00:52:29.424 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:52:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:52:31.122 INFO:teuthology.orchestra.run.smithi098.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 4.6 MB/s | 35 MB 00:07 2024-09-18T00:52:31.122 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:52:31.122 INFO:teuthology.orchestra.run.smithi098.stdout:Total 10 MB/s | 79 MB 00:07 2024-09-18T00:52:31.807 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T00:52:32.140 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T00:52:32.140 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T00:52:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:52:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:52:33.084 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T00:52:33.084 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T00:52:33.282 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-18T00:52:35.213 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T00:52:35.213 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T00:52:35.308 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T00:52:35.316 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T00:52:35.316 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-18T00:52:35.316 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T00:52:35.316 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T00:52:35.316 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T00:52:35.316 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-18T00:52:35.317 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:52:35.318 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 79 M 2024-09-18T00:52:35.318 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 126 M 2024-09-18T00:52:35.318 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T00:52:35.433 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 421 kB/s | 36 kB 00:00 2024-09-18T00:52:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T00:52:35.825 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-18T00:52:36.157 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:52:36.249 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:52:36.258 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-18T00:52:36.625 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-18T00:52:36.626 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:52:36.626 INFO:teuthology.orchestra.run.smithi164.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-18T00:52:37.308 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T00:52:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:52:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:52:37.675 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T00:52:37.675 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T00:52:38.614 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T00:52:38.615 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T00:52:39.045 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:52:39.177 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:52:41.048 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T00:52:41.771 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:52:41.859 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:52:42.716 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:52:44.011 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:52:44.053 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:52:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:52:44.770 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:52:48.894 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:52:49.595 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:52:49.704 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:52:54.535 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:22.115 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:53:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:53:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T00:53:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:53:23.269 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T00:53:23.624 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:53:23.715 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:23.716 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:53:23.716 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T00:53:23.716 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:53:23.716 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:53:23.716 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T00:53:23.716 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:53:23.717 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:53:23.775 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:23.776 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:53:23.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-18T00:53:23.808 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-18T00:53:23.808 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-18T00:53:23.808 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T00:53:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:53:23.810 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T00:53:23.811 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T00:53:23.811 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-508.el9.x86_64 2024-09-18T00:53:23.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T00:53:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-18T00:53:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T00:53:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-18T00:53:25.774 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-18T00:53:25.789 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.YhnebRKvaR 2024-09-18T00:53:25.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YhnebRKvaR 2024-09-18T00:53:25.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.YhnebRKvaR 2024-09-18T00:53:26.014 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.YhnebRKvaR is 6KB 2024-09-18T00:53:26.064 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.YhnebRKvaR 2024-09-18T00:53:26.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T00:53:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T00:53:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T00:53:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T00:53:26.141 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T00:53:26.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T00:53:26.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-18T00:53:27.284 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:27.325 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:27.674 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:27.675 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:53:27.675 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:53:27.675 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:53:28.515 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:28.515 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:53:28.515 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T00:53:28.515 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:28.516 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T00:53:28.516 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T00:53:28.516 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T00:53:28.516 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:53:28.516 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T00:53:28.870 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:53:28.952 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:28.952 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:53:28.952 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T00:53:28.952 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:53:28.952 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:53:28.952 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T00:53:28.952 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi098.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:53:28.953 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:53:29.010 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:29.010 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:53:29.010 DEBUG:teuthology.orchestra.run.smithi098:> sudo rpm -qi grub2-tools 2024-09-18T00:53:29.052 INFO:teuthology.orchestra.run.smithi098.stdout:Name : grub2-tools 2024-09-18T00:53:29.052 INFO:teuthology.orchestra.run.smithi098.stdout:Epoch : 1 2024-09-18T00:53:29.052 INFO:teuthology.orchestra.run.smithi098.stdout:Version : 2.06 2024-09-18T00:53:29.052 INFO:teuthology.orchestra.run.smithi098.stdout:Release : 82.el9 2024-09-18T00:53:29.052 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: x86_64 2024-09-18T00:53:29.052 INFO:teuthology.orchestra.run.smithi098.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Group : Unspecified 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Size : 8274670 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:License : GPLv3+ 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Packager : builder@centos.org 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Vendor : CentOS 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Summary : Support tools for GRUB. 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Description : 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:53:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T00:53:29.054 INFO:teuthology.orchestra.run.smithi098.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T00:53:29.054 INFO:teuthology.orchestra.run.smithi098.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T00:53:29.054 INFO:teuthology.orchestra.run.smithi098.stdout:hardware devices. 2024-09-18T00:53:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:53:29.054 INFO:teuthology.orchestra.run.smithi098.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T00:53:29.055 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T00:53:29.055 INFO:teuthology.task.kernel:Updating grub on smithi098 to boot 5.14.0-508.el9.x86_64 2024-09-18T00:53:29.055 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T00:53:29.708 INFO:teuthology.orchestra.run.smithi098.stderr:Generating grub configuration file ... 2024-09-18T00:53:31.017 INFO:teuthology.orchestra.run.smithi098.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T00:53:31.044 INFO:teuthology.orchestra.run.smithi098.stderr:done 2024-09-18T00:53:31.046 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-18T00:53:31.061 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.V2uxlsMWqB 2024-09-18T00:53:31.062 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.V2uxlsMWqB 2024-09-18T00:53:31.128 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.V2uxlsMWqB 2024-09-18T00:53:31.266 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.V2uxlsMWqB is 6KB 2024-09-18T00:53:31.313 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.V2uxlsMWqB 2024-09-18T00:53:31.328 DEBUG:teuthology.orchestra.run.smithi098:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T00:53:31.399 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T00:53:31.399 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T00:53:31.399 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T00:53:31.399 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T00:53:31.400 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T00:53:31.566 DEBUG:teuthology.orchestra.run.smithi098:> sudo shutdown -r now 2024-09-18T00:53:32.795 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:33.124 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T00:53:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T00:53:33.125 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T00:53:34.022 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T00:53:34.022 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:53:34.022 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T00:53:34.022 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:34.022 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T00:53:34.022 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T00:53:34.023 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T00:53:34.023 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:53:34.023 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T00:53:34.382 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:53:34.470 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:34.470 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:53:34.470 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T00:53:34.470 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T00:53:34.470 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T00:53:34.471 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-18T00:53:34.471 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T00:53:34.471 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T00:53:34.530 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T00:53:34.530 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T00:53:34.530 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 82.el9 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8274670 2024-09-18T00:53:34.575 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-09-18T00:53:34.576 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-09-18T00:53:34.577 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:53:34.577 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T00:53:34.577 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T00:53:34.577 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T00:53:34.577 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-09-18T00:53:34.577 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:53:34.577 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T00:53:34.578 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T00:53:34.578 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-508.el9.x86_64 2024-09-18T00:53:34.578 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T00:53:35.268 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-18T00:53:36.644 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T00:53:36.679 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-18T00:53:36.681 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-18T00:53:36.696 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.dG2rc1GSpl 2024-09-18T00:53:36.697 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dG2rc1GSpl 2024-09-18T00:53:36.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.dG2rc1GSpl 2024-09-18T00:53:36.906 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.dG2rc1GSpl is 6KB 2024-09-18T00:53:36.954 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.dG2rc1GSpl 2024-09-18T00:53:36.969 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T00:53:37.031 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T00:53:37.031 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T00:53:37.032 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T00:53:37.032 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T00:53:37.033 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T00:53:37.195 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-18T00:53:56.303 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T00:53:56.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-18T00:53:56.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:01.573 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T00:54:01.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-18T00:54:01.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:07.201 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-18T00:54:07.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-18T00:54:07.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:14.845 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-18T00:54:23.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-18T00:54:23.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:25.597 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-18T00:54:34.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-18T00:54:34.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:37.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-18T00:54:42.237 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-18T00:54:49.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-18T00:54:49.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:54.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-18T00:54:54.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:54:57.877 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-18T00:54:58.297 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-18T00:54:58.298 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-18T00:54:58.298 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-18T00:54:58.313 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-508.el9.x86_64 2024-09-18T00:54:58.313 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T00:54:58.313 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:54:58.314 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T00:54:59.314 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T00:54:59.314 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:54:59.425 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-18T00:54:59.444 DEBUG:teuthology.parallel:result is None 2024-09-18T00:55:01.576 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T00:55:08.097 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-18T00:55:10.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-18T00:55:10.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:55:11.075 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-18T00:55:11.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-18T00:55:11.510 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-18T00:55:11.510 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-18T00:55:11.524 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-508.el9.x86_64 2024-09-18T00:55:11.525 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T00:55:11.525 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:55:11.525 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T00:55:12.526 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T00:55:12.526 DEBUG:teuthology.orchestra.run.smithi098:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:55:12.610 INFO:teuthology.orchestra.run.smithi098.stdout:ttyS1 2024-09-18T00:55:12.635 DEBUG:teuthology.parallel:result is None 2024-09-18T00:55:23.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-18T00:55:23.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:55:23.625 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-18T00:55:24.015 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-18T00:55:24.015 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-18T00:55:24.016 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-18T00:55:24.030 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-508.el9.x86_64 2024-09-18T00:55:24.030 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T00:55:24.031 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T00:55:24.031 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T00:55:25.032 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-18T00:55:25.032 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T00:55:25.141 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-09-18T00:55:25.171 DEBUG:teuthology.parallel:result is None 2024-09-18T00:55:25.171 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T00:55:25.177 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T00:55:25.177 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T00:55:25.181 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T00:55:25.184 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T00:55:25.201 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T00:55:25.215 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T00:55:25.226 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T00:55:25.226 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T00:55:25.242 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T00:55:25.244 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T00:55:25.281 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T00:55:25.288 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T00:55:25.288 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T00:55:25.308 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T00:55:25.319 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T00:55:25.349 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:55:25.351 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:55:25.360 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:55:25.363 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:55:25.382 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:55:25.393 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T00:55:25.395 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T00:55:25.401 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T00:55:25.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T00:55:25.405 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T00:55:25.407 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T00:55:25.432 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T00:55:25.442 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T00:55:25.442 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T00:55:25.473 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T00:55:25.475 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T00:55:25.490 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:55:25.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:55:25.623 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:55:25.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:55:25.748 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T00:55:25.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T00:55:25.815 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:55:25.862 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:55:25.909 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:55:25.993 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:55:26.034 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T00:55:26.034 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T00:55:26.099 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:55:26.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T00:55:26.187 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:55:26.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T00:55:26.307 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T00:55:26.307 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T00:55:26.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-18T00:55:26.374 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-09-18T00:55:26.376 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-18T00:55:26.415 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T00:55:26.432 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T00:55:26.462 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T00:55:26.942 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T00:55:26.949 INFO:teuthology.task.internal:Starting timer... 2024-09-18T00:55:26.949 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T00:55:26.961 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T00:55:26.972 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-18T00:55:26.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-18T00:55:27.039 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-18T00:55:27.041 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-09-18T00:55:27.109 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-09-18T00:55:27.111 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-09-18T00:55:27.154 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-09-18T00:55:27.156 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T00:55:27.157 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-18T00:55:27.192 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-18T00:55:27.192 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-09-18T00:55:27.211 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-09-18T00:55:27.211 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-09-18T00:55:27.229 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-09-18T00:55:27.229 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-09-18T00:55:27.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T00:55:27.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:55:27.261 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T00:55:27.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:55:27.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T00:55:27.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T00:55:27.315 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T00:55:27.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-18T00:55:27.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-09-18T00:55:27.380 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-09-18T00:55:27.413 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T00:55:27.422 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T00:55:27.432 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T00:55:27.433 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi098.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T00:59:13.115 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-09-18T00:59:13.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-18T00:59:13.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:59:13.196 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-18T00:59:13.273 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-18T00:59:13.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-18T00:59:13.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:59:13.348 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-18T00:59:13.433 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-18T00:59:13.433 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-18T00:59:13.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T00:59:13.508 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-18T00:59:13.590 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-18T00:59:13.590 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T00:59:13.602 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T00:59:13.602 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T00:59:13.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:59:13.605 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T00:59:13.605 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:59:13.608 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T00:59:13.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T00:59:13.637 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T00:59:13.637 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T00:59:13.651 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T00:59:13.654 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T00:59:13.662 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T00:59:13.676 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-09-18T00:59:13.677 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T00:59:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-18T00:59:13.688 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-09-18T00:59:13.694 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-18T00:59:13.702 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T00:59:13.703 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-09-18T00:59:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T00:59:13.714 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-09-18T00:59:13.716 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T00:59:13.725 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T00:59:13.730 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T00:59:13.744 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T00:59:13.772 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-18T00:59:13.775 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T00:59:13.775 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-18T00:59:13.775 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.776 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.776 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.776 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.788 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-18T00:59:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T00:59:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-18T00:59:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.792 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.802 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-18T00:59:13.806 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T00:59:13.806 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-18T00:59:13.806 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.806 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.806 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.806 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T00:59:13.807 INFO:teuthology.run_tasks:Running task install... 2024-09-18T00:59:13.817 DEBUG:teuthology.task.install:project ceph 2024-09-18T00:59:13.817 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-18T00:59:13.817 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-18T00:59:13.817 INFO:teuthology.task.install:Using flavor: default 2024-09-18T00:59:13.822 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T00:59:13.822 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-18T00:59:13.823 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-18T00:59:13.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:59:13.826 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-18T00:59:13.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:59:13.827 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-18T00:59:13.827 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T00:59:13.971 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-18T00:59:13.971 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-18T00:59:13.974 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-18T00:59:13.974 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-18T00:59:13.983 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-18T00:59:13.983 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-18T00:59:14.081 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T00:59:14.081 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T00:59:14.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T00:59:14.088 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T00:59:14.088 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T00:59:14.088 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T00:59:14.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T00:59:14.098 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T00:59:14.098 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T00:59:14.111 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-18T00:59:14.111 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T00:59:14.114 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-18T00:59:14.114 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T00:59:14.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-18T00:59:14.124 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T00:59:14.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T00:59:14.180 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T00:59:14.190 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T00:59:14.255 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T00:59:14.259 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T00:59:14.265 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T00:59:14.320 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-09-18T00:59:14.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-09-18T00:59:14.325 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-18T00:59:14.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-18T00:59:14.329 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-09-18T00:59:14.331 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-09-18T00:59:14.555 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-09-18T00:59:14.560 INFO:teuthology.orchestra.run.smithi098.stdout:45 files removed 2024-09-18T00:59:14.562 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-18T00:59:14.581 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw 2024-09-18T00:59:14.586 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw 2024-09-18T00:59:14.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-09-18T00:59:15.125 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 322 kB/s | 84 kB 00:00 2024-09-18T00:59:15.145 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-18T00:59:15.168 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-18T00:59:15.412 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-18T00:59:15.418 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-18T00:59:15.452 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-18T00:59:15.671 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-18T00:59:15.718 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 7.2 kB/s | 1.9 kB 00:00 2024-09-18T00:59:15.727 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-18T00:59:17.402 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.3 MB 00:01 2024-09-18T00:59:17.493 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-18T00:59:17.554 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-18T00:59:22.351 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 20 MB 00:03 2024-09-18T00:59:22.677 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 20 MB 00:04 2024-09-18T00:59:26.195 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-18T00:59:28.178 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-18T00:59:29.140 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.5 MB 00:02 2024-09-18T00:59:29.829 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-18T00:59:30.857 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 19 kB 00:00 2024-09-18T00:59:31.696 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-18T00:59:32.409 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 23 MB 00:02 2024-09-18T00:59:33.236 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 10 MB/s | 23 MB 00:02 2024-09-18T00:59:33.429 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-18T00:59:35.125 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-18T00:59:37.872 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-18T00:59:38.625 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-18T00:59:40.124 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T00:59:40.140 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T00:59:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T00:59:40.140 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T00:59:40.140 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T00:59:40.140 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-18T00:59:40.140 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T00:59:40.141 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-18T00:59:40.142 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T00:59:40.143 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T00:59:40.144 INFO:teuthology.orchestra.run.smithi164.stdout:Install 30 Packages 2024-09-18T00:59:40.144 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 2 Packages 2024-09-18T00:59:40.144 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:59:40.144 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 60 M 2024-09-18T00:59:40.144 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T00:59:40.822 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-09-18T00:59:40.859 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T00:59:40.875 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-18T00:59:40.876 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T00:59:40.877 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T00:59:40.878 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout:Install 30 Packages 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 2 Packages 2024-09-18T00:59:40.879 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 60 M 2024-09-18T00:59:40.880 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T00:59:40.932 INFO:teuthology.orchestra.run.smithi164.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 8.6 MB/s | 5.4 MB 00:00 2024-09-18T00:59:40.990 INFO:teuthology.orchestra.run.smithi164.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 433 kB/s | 25 kB 00:00 2024-09-18T00:59:41.124 INFO:teuthology.orchestra.run.smithi164.stdout:(3/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 13 MB/s | 11 MB 00:00 2024-09-18T00:59:41.166 INFO:teuthology.orchestra.run.smithi164.stdout:(4/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 4.2 MB/s | 751 kB 00:00 2024-09-18T00:59:41.199 INFO:teuthology.orchestra.run.smithi164.stdout:(5/32): libradosstriper1-19.1.1-330.gc2ddf972.e 6.6 MB/s | 506 kB 00:00 2024-09-18T00:59:41.250 INFO:teuthology.orchestra.run.smithi164.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 905 kB/s | 45 kB 00:00 2024-09-18T00:59:41.459 INFO:teuthology.orchestra.run.smithi164.stdout:(7/32): ceph-common-19.1.1-330.gc2ddf972.el9.x8 18 MB/s | 21 MB 00:01 2024-09-18T00:59:41.483 INFO:teuthology.orchestra.run.smithi164.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 583 kB/s | 136 kB 00:00 2024-09-18T00:59:41.509 INFO:teuthology.orchestra.run.smithi164.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 3.2 MB/s | 162 kB 00:00 2024-09-18T00:59:41.543 INFO:teuthology.orchestra.run.smithi164.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 5.3 MB/s | 320 kB 00:00 2024-09-18T00:59:41.576 INFO:teuthology.orchestra.run.smithi164.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 4.4 MB/s | 300 kB 00:00 2024-09-18T00:59:41.660 INFO:teuthology.orchestra.run.smithi164.stdout:(12/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 5.3 MB 00:00 2024-09-18T00:59:41.686 INFO:teuthology.orchestra.run.smithi164.stdout:(13/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 697 kB/s | 99 kB 00:00 2024-09-18T00:59:41.777 INFO:teuthology.orchestra.run.smithi164.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 532 kB/s | 107 kB 00:00 2024-09-18T00:59:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-18T00:59:41.828 INFO:teuthology.orchestra.run.smithi164.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 958 kB/s | 160 kB 00:00 2024-09-18T00:59:41.878 INFO:teuthology.orchestra.run.smithi164.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-18T00:59:41.920 INFO:teuthology.orchestra.run.smithi164.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-18T00:59:41.926 INFO:teuthology.orchestra.run.smithi098.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 8.5 MB/s | 5.4 MB 00:00 2024-09-18T00:59:41.945 INFO:teuthology.orchestra.run.smithi164.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-18T00:59:41.971 INFO:teuthology.orchestra.run.smithi164.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 715 kB/s | 36 kB 00:00 2024-09-18T00:59:41.996 INFO:teuthology.orchestra.run.smithi164.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-18T00:59:42.021 INFO:teuthology.orchestra.run.smithi164.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-09-18T00:59:42.043 INFO:teuthology.orchestra.run.smithi098.stdout:(2/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 14 MB/s | 11 MB 00:00 2024-09-18T00:59:42.069 INFO:teuthology.orchestra.run.smithi098.stdout:(3/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 177 kB/s | 25 kB 00:00 2024-09-18T00:59:42.088 INFO:teuthology.orchestra.run.smithi164.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 501 kB/s | 26 kB 00:00 2024-09-18T00:59:42.102 INFO:teuthology.orchestra.run.smithi098.stdout:(4/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 13 MB/s | 751 kB 00:00 2024-09-18T00:59:42.113 INFO:teuthology.orchestra.run.smithi164.stdout:(24/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-18T00:59:42.128 INFO:teuthology.orchestra.run.smithi098.stdout:(5/32): libradosstriper1-19.1.1-330.gc2ddf972.e 8.4 MB/s | 506 kB 00:00 2024-09-18T00:59:42.139 INFO:teuthology.orchestra.run.smithi164.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-18T00:59:42.164 INFO:teuthology.orchestra.run.smithi164.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-18T00:59:42.195 INFO:teuthology.orchestra.run.smithi098.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 677 kB/s | 45 kB 00:00 2024-09-18T00:59:42.287 INFO:teuthology.orchestra.run.smithi098.stdout:(7/32): python3-ceph-common-19.1.1-330.gc2ddf97 1.5 MB/s | 136 kB 00:00 2024-09-18T00:59:42.362 INFO:teuthology.orchestra.run.smithi098.stdout:(8/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_64 21 MB/s | 5.3 MB 00:00 2024-09-18T00:59:42.387 INFO:teuthology.orchestra.run.smithi098.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 1.6 MB/s | 162 kB 00:00 2024-09-18T00:59:42.391 INFO:teuthology.orchestra.run.smithi164.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 844 kB/s | 191 kB 00:00 2024-09-18T00:59:42.412 INFO:teuthology.orchestra.run.smithi098.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 6.2 MB/s | 320 kB 00:00 2024-09-18T00:59:42.438 INFO:teuthology.orchestra.run.smithi098.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-18T00:59:42.463 INFO:teuthology.orchestra.run.smithi098.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-18T00:59:42.465 INFO:teuthology.orchestra.run.smithi164.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.5 MB/s | 839 kB 00:00 2024-09-18T00:59:42.582 INFO:teuthology.orchestra.run.smithi164.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-09-18T00:59:42.649 INFO:teuthology.orchestra.run.smithi164.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-09-18T00:59:42.680 INFO:teuthology.orchestra.run.smithi098.stdout:(13/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 15 MB/s | 21 MB 00:01 2024-09-18T00:59:42.749 INFO:teuthology.orchestra.run.smithi164.stdout:(31/32): librados2-19.1.1-330.gc2ddf972.el9.x86 12 MB/s | 3.4 MB 00:00 2024-09-18T00:59:42.814 INFO:teuthology.orchestra.run.smithi098.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 340 kB/s | 45 kB 00:00 2024-09-18T00:59:42.839 INFO:teuthology.orchestra.run.smithi098.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 427 kB/s | 160 kB 00:00 2024-09-18T00:59:42.864 INFO:teuthology.orchestra.run.smithi098.stdout:(16/32): boost-program-options-1.75.0-8.el9.x86 251 kB/s | 107 kB 00:00 2024-09-18T00:59:42.874 INFO:teuthology.orchestra.run.smithi164.stdout:(32/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 3.2 MB 00:00 2024-09-18T00:59:42.879 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:59:42.880 INFO:teuthology.orchestra.run.smithi164.stdout:Total 22 MB/s | 60 MB 00:02 2024-09-18T00:59:42.914 INFO:teuthology.orchestra.run.smithi098.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-18T00:59:42.956 INFO:teuthology.orchestra.run.smithi098.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-18T00:59:42.985 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T00:59:42.998 INFO:teuthology.orchestra.run.smithi098.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-18T00:59:43.020 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T00:59:43.020 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T00:59:43.024 INFO:teuthology.orchestra.run.smithi098.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 535 kB/s | 36 kB 00:00 2024-09-18T00:59:43.049 INFO:teuthology.orchestra.run.smithi098.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-18T00:59:43.074 INFO:teuthology.orchestra.run.smithi098.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-18T00:59:43.133 INFO:teuthology.orchestra.run.smithi098.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-18T00:59:43.158 INFO:teuthology.orchestra.run.smithi098.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-18T00:59:43.184 INFO:teuthology.orchestra.run.smithi098.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 274 kB/s | 26 kB 00:00 2024-09-18T00:59:43.209 INFO:teuthology.orchestra.run.smithi098.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-18T00:59:43.209 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T00:59:43.212 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T00:59:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-18T00:59:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-18T00:59:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:59:43.215 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T00:59:43.216 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T00:59:43.216 INFO:teuthology.orchestra.run.smithi031.stdout:Install 30 Packages 2024-09-18T00:59:43.216 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-09-18T00:59:43.216 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:59:43.216 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 60 M 2024-09-18T00:59:43.216 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T00:59:43.251 INFO:teuthology.orchestra.run.smithi098.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-18T00:59:43.359 INFO:teuthology.orchestra.run.smithi098.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-09-18T00:59:43.476 INFO:teuthology.orchestra.run.smithi098.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-09-18T00:59:43.610 INFO:teuthology.orchestra.run.smithi098.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-09-18T00:59:43.677 INFO:teuthology.orchestra.run.smithi098.stdout:(31/32): librados2-19.1.1-330.gc2ddf972.el9.x86 11 MB/s | 3.4 MB 00:00 2024-09-18T00:59:43.681 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T00:59:43.682 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T00:59:43.765 INFO:teuthology.orchestra.run.smithi031.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-18T00:59:43.815 INFO:teuthology.orchestra.run.smithi031.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 504 kB/s | 25 kB 00:00 2024-09-18T00:59:43.827 INFO:teuthology.orchestra.run.smithi098.stdout:(32/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 9.0 MB/s | 3.2 MB 00:00 2024-09-18T00:59:43.832 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:59:43.832 INFO:teuthology.orchestra.run.smithi098.stdout:Total 20 MB/s | 60 MB 00:02 2024-09-18T00:59:43.899 INFO:teuthology.orchestra.run.smithi031.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 8.8 MB/s | 751 kB 00:00 2024-09-18T00:59:43.934 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T00:59:43.970 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T00:59:43.970 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T00:59:43.990 INFO:teuthology.orchestra.run.smithi031.stdout:(4/32): libradosstriper1-19.1.1-330.gc2ddf972.e 5.4 MB/s | 506 kB 00:00 2024-09-18T00:59:44.099 INFO:teuthology.orchestra.run.smithi031.stdout:(5/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 13 MB/s | 11 MB 00:00 2024-09-18T00:59:44.158 INFO:teuthology.orchestra.run.smithi031.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 776 kB/s | 45 kB 00:00 2024-09-18T00:59:44.250 INFO:teuthology.orchestra.run.smithi031.stdout:(7/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_64 21 MB/s | 5.3 MB 00:00 2024-09-18T00:59:44.275 INFO:teuthology.orchestra.run.smithi031.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 1.1 MB/s | 136 kB 00:00 2024-09-18T00:59:44.300 INFO:teuthology.orchestra.run.smithi031.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 3.2 MB/s | 162 kB 00:00 2024-09-18T00:59:44.334 INFO:teuthology.orchestra.run.smithi031.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 5.3 MB/s | 320 kB 00:00 2024-09-18T00:59:44.359 INFO:teuthology.orchestra.run.smithi031.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-18T00:59:44.393 INFO:teuthology.orchestra.run.smithi031.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-18T00:59:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 712 kB/s | 107 kB 00:00 2024-09-18T00:59:44.653 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T00:59:44.653 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T00:59:44.710 INFO:teuthology.orchestra.run.smithi031.stdout:(14/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 15 MB/s | 21 MB 00:01 2024-09-18T00:59:44.735 INFO:teuthology.orchestra.run.smithi031.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 468 kB/s | 160 kB 00:00 2024-09-18T00:59:44.760 INFO:teuthology.orchestra.run.smithi031.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 181 kB/s | 45 kB 00:00 2024-09-18T00:59:44.811 INFO:teuthology.orchestra.run.smithi031.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 845 kB/s | 42 kB 00:00 2024-09-18T00:59:44.836 INFO:teuthology.orchestra.run.smithi031.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-18T00:59:44.886 INFO:teuthology.orchestra.run.smithi031.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-18T00:59:44.911 INFO:teuthology.orchestra.run.smithi031.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 477 kB/s | 36 kB 00:00 2024-09-18T00:59:44.937 INFO:teuthology.orchestra.run.smithi031.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-18T00:59:44.987 INFO:teuthology.orchestra.run.smithi031.stdout:(22/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-18T00:59:45.045 INFO:teuthology.orchestra.run.smithi031.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.2 MB/s | 308 kB 00:00 2024-09-18T00:59:45.071 INFO:teuthology.orchestra.run.smithi031.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 318 kB/s | 26 kB 00:00 2024-09-18T00:59:45.096 INFO:teuthology.orchestra.run.smithi031.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-18T00:59:45.122 INFO:teuthology.orchestra.run.smithi031.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-18T00:59:45.143 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T00:59:45.188 INFO:teuthology.orchestra.run.smithi031.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-18T00:59:45.213 INFO:teuthology.orchestra.run.smithi031.stdout:(28/32): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-18T00:59:45.225 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-18T00:59:45.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-18T00:59:45.297 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-18T00:59:45.319 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T00:59:45.346 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T00:59:45.372 INFO:teuthology.orchestra.run.smithi031.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 22 MB/s | 3.4 MB 00:00 2024-09-18T00:59:45.439 INFO:teuthology.orchestra.run.smithi031.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-18T00:59:45.514 INFO:teuthology.orchestra.run.smithi031.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 22 MB/s | 3.2 MB 00:00 2024-09-18T00:59:45.538 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T00:59:45.581 INFO:teuthology.orchestra.run.smithi031.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-18T00:59:45.586 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T00:59:45.586 INFO:teuthology.orchestra.run.smithi031.stdout:Total 25 MB/s | 60 MB 00:02 2024-09-18T00:59:45.600 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T00:59:45.691 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T00:59:45.732 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T00:59:45.990 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-18T00:59:46.048 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T00:59:46.140 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T00:59:46.172 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T00:59:46.242 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-18T00:59:46.253 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-18T00:59:46.275 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T00:59:46.282 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-18T00:59:46.308 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-18T00:59:46.330 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T00:59:46.340 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T00:59:46.356 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T00:59:46.379 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-18T00:59:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T00:59:46.429 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T00:59:46.435 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-18T00:59:46.506 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T00:59:46.516 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T00:59:46.544 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T00:59:46.568 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T00:59:46.608 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T00:59:46.651 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-18T00:59:46.771 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-18T00:59:46.811 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-18T00:59:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-18T00:59:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-18T00:59:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-18T00:59:46.954 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-18T00:59:46.959 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-18T00:59:47.017 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T00:59:47.110 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T00:59:47.199 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-18T00:59:47.236 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T00:59:47.267 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-18T00:59:47.299 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T00:59:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T00:59:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-18T00:59:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T00:59:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-18T00:59:47.413 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-18T00:59:47.450 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-18T00:59:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T00:59:47.488 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T00:59:47.552 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T00:59:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-18T00:59:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-18T00:59:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-18T00:59:47.779 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-18T00:59:47.819 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-18T00:59:47.848 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-18T00:59:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-18T00:59:47.930 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-18T00:59:47.957 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T00:59:47.961 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T00:59:48.036 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T00:59:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-18T00:59:48.109 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-18T00:59:48.139 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-18T00:59:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-18T00:59:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-18T00:59:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-18T00:59:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T00:59:48.261 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T00:59:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-18T00:59:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T00:59:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-18T00:59:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-18T00:59:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T00:59:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-18T00:59:48.921 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-18T00:59:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-18T00:59:48.947 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T00:59:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T00:59:49.019 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T00:59:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T00:59:49.095 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-18T00:59:49.125 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-18T00:59:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-18T00:59:49.198 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T00:59:49.261 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-18T00:59:49.298 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-18T00:59:49.342 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-18T00:59:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T00:59:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T00:59:49.514 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T00:59:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-18T00:59:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-18T00:59:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-18T00:59:49.751 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-18T00:59:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-18T00:59:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-18T00:59:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-18T00:59:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-18T00:59:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T00:59:50.322 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-18T00:59:50.352 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-18T00:59:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-18T00:59:50.438 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:50.494 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-18T00:59:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T00:59:51.008 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-18T00:59:51.060 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:51.085 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-18T00:59:51.108 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T00:59:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-18T00:59:51.192 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T00:59:51.192 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T00:59:51.192 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T00:59:51.192 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T00:59:51.226 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T00:59:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:51.471 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:52.038 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:52.075 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T00:59:52.158 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T00:59:52.158 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T00:59:52.158 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T00:59:52.158 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T00:59:52.193 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T00:59:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:54.136 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-18T00:59:54.168 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T00:59:54.264 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-18T00:59:54.264 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T00:59:54.265 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T00:59:54.265 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T00:59:54.302 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:00:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:00:01.378 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-09-18T01:00:01.378 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-09-18T01:00:01.378 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-09-18T01:00:01.379 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-09-18T01:00:01.379 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-09-18T01:00:01.379 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-09-18T01:00:01.379 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-09-18T01:00:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:00:02.175 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-09-18T01:00:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-09-18T01:00:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-09-18T01:00:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-09-18T01:00:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-09-18T01:00:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-09-18T01:00:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-09-18T01:00:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:02.386 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:00:02.411 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:00:02.412 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:02.412 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T01:00:02.412 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:00:02.412 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:00:02.412 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:02.413 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:00:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:00:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:03.190 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:00:03.216 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:00:03.216 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:03.216 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T01:00:03.216 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:00:03.216 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:00:03.216 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:03.218 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:00:03.309 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:00:03.309 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-18T01:00:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-18T01:00:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-18T01:00:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-18T01:00:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-18T01:00:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-18T01:00:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-18T01:00:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:00:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-18T01:00:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-18T01:00:04.542 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-18T01:00:04.543 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-18T01:00:04.544 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-09-18T01:00:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T01:00:04.638 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.639 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:04.640 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-18T01:00:04.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:05.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-test 2024-09-18T01:00:05.534 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:05.534 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:05.534 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-09-18T01:00:05.534 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.534 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.534 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T01:00:05.535 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:05.537 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-18T01:00:05.537 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-18T01:00:05.537 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-18T01:00:05.537 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T01:00:05.537 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T01:00:05.537 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:05.537 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:05.586 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:28 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:05.718 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:05.718 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:05.719 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-18T01:00:05.720 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:05.720 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 49 M 2024-09-18T01:00:05.720 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 209 M 2024-09-18T01:00:05.720 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:05.808 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:00:05.835 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-18T01:00:05.835 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:05.835 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T01:00:05.836 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:00:05.836 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T01:00:05.836 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:05.838 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:00:05.914 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-test 2024-09-18T01:00:05.927 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-18T01:00:05.928 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:06.044 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 778 kB/s | 163 kB 00:00 2024-09-18T01:00:06.485 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:28 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:06.620 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T01:00:06.621 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T01:00:06.622 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:06.622 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:00:06.622 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:06.622 INFO:teuthology.orchestra.run.smithi098.stdout:Install 4 Packages 2024-09-18T01:00:06.622 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:06.622 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 49 M 2024-09-18T01:00:06.623 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 209 M 2024-09-18T01:00:06.623 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:00:06.661 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-09-18T01:00:06.744 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 334 kB/s | 304 kB 00:00 2024-09-18T01:00:06.806 INFO:teuthology.orchestra.run.smithi098.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-18T01:00:06.848 INFO:teuthology.orchestra.run.smithi098.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-18T01:00:06.990 INFO:teuthology.orchestra.run.smithi098.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 508 kB/s | 163 kB 00:00 2024-09-18T01:00:07.121 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:07.121 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-18T01:00:07.122 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-18T01:00:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-18T01:00:07.124 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-18T01:00:07.124 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-18T01:00:07.124 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-18T01:00:07.124 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-18T01:00:07.124 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-18T01:00:07.311 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 33 MB/s | 49 MB 00:01 2024-09-18T01:00:07.314 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:07.314 INFO:teuthology.orchestra.run.smithi164.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-18T01:00:07.342 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:07.362 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:07.362 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:07.750 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:07.750 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:00:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-18T01:00:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T01:00:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-18T01:00:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-18T01:00:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-18T01:00:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T01:00:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T01:00:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:08.065 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:08.274 INFO:teuthology.orchestra.run.smithi098.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 30 MB/s | 49 MB 00:01 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:08.276 INFO:teuthology.orchestra.run.smithi098.stdout:Total 30 MB/s | 49 MB 00:01 2024-09-18T01:00:08.278 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:00:08.303 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:00:08.323 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:00:08.360 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T01:00:08.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-09-18T01:00:08.411 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T01:00:08.438 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:00:08.711 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:00:08.712 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:00:08.982 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:28 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:09.130 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:09.130 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:09.130 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-18T01:00:09.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 49 M 2024-09-18T01:00:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 209 M 2024-09-18T01:00:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:00:09.241 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:00:09.325 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T01:00:09.375 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T01:00:09.400 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:00:09.631 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 740 kB/s | 163 kB 00:00 2024-09-18T01:00:10.215 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 378 kB/s | 304 kB 00:00 2024-09-18T01:00:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 104 kB/s | 64 kB 00:00 2024-09-18T01:00:10.924 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-18T01:00:10.927 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:10.927 INFO:teuthology.orchestra.run.smithi031.stdout:Total 27 MB/s | 49 MB 00:01 2024-09-18T01:00:10.955 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:00:10.978 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:00:10.978 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:00:11.391 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:00:11.391 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:00:11.942 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:00:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T01:00:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T01:00:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:00:13.052 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:00:13.154 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:00:14.052 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:00:14.124 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:00:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:00:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-18T01:00:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-18T01:00:15.629 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T01:00:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T01:00:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:15.980 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:15.980 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T01:00:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T01:00:15.981 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:15.981 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:16.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph 2024-09-18T01:00:16.642 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:00:16.642 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-18T01:00:16.642 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-18T01:00:16.642 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T01:00:16.860 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:39 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:17.002 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:17.004 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:17.004 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:00:17.004 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:17.004 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T01:00:17.005 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T01:00:17.006 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T01:00:17.007 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T01:00:17.008 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:17.008 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:17.008 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:17.008 INFO:teuthology.orchestra.run.smithi164.stdout:Install 37 Packages 2024-09-18T01:00:17.008 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:17.008 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 30 M 2024-09-18T01:00:17.009 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 108 M 2024-09-18T01:00:17.009 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-18T01:00:17.021 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T01:00:17.021 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:17.021 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:00:17.021 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:17.022 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:17.022 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T01:00:17.022 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T01:00:17.022 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:17.022 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:17.102 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:00:17.270 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph 2024-09-18T01:00:17.567 INFO:teuthology.orchestra.run.smithi164.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-18T01:00:17.768 INFO:teuthology.orchestra.run.smithi164.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-18T01:00:17.818 INFO:teuthology.orchestra.run.smithi164.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.4 MB/s | 2.4 MB 00:00 2024-09-18T01:00:17.879 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:39 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:17.918 INFO:teuthology.orchestra.run.smithi164.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.4 MB/s | 249 kB 00:00 2024-09-18T01:00:17.985 INFO:teuthology.orchestra.run.smithi164.stdout:(5/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-18T01:00:18.030 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:18.033 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:18.033 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:00:18.033 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:18.033 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:00:18.033 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T01:00:18.034 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T01:00:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T01:00:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T01:00:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T01:00:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T01:00:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T01:00:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T01:00:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T01:00:18.036 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:18.037 INFO:teuthology.orchestra.run.smithi098.stdout:Install 37 Packages 2024-09-18T01:00:18.038 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:18.038 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 30 M 2024-09-18T01:00:18.038 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 108 M 2024-09-18T01:00:18.038 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:00:18.317 INFO:teuthology.orchestra.run.smithi098.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 50 kB/s | 6.5 kB 00:00 2024-09-18T01:00:18.327 INFO:teuthology.orchestra.run.smithi164.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-18T01:00:18.543 INFO:teuthology.orchestra.run.smithi098.stdout:(2/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.7 MB/s | 2.4 MB 00:00 2024-09-18T01:00:18.584 INFO:teuthology.orchestra.run.smithi098.stdout:(3/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 3.8 MB/s | 1.5 MB 00:00 2024-09-18T01:00:18.602 INFO:teuthology.orchestra.run.smithi164.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 370 kB/s | 253 kB 00:00 2024-09-18T01:00:18.693 INFO:teuthology.orchestra.run.smithi098.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.3 MB/s | 249 kB 00:00 2024-09-18T01:00:18.720 INFO:teuthology.orchestra.run.smithi164.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-18T01:00:18.760 INFO:teuthology.orchestra.run.smithi098.stdout:(5/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-18T01:00:18.778 INFO:teuthology.orchestra.run.smithi164.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 236 kB/s | 106 kB 00:00 2024-09-18T01:00:18.828 INFO:teuthology.orchestra.run.smithi164.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 125 kB 00:00 2024-09-18T01:00:18.912 INFO:teuthology.orchestra.run.smithi164.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2024-09-18T01:00:19.012 INFO:teuthology.orchestra.run.smithi164.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-18T01:00:19.063 INFO:teuthology.orchestra.run.smithi164.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-09-18T01:00:19.086 INFO:teuthology.orchestra.run.smithi098.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-18T01:00:19.096 INFO:teuthology.orchestra.run.smithi164.stdout:(14/37): python3-urllib3-1.26.5-6.el9.noarch.rp 676 kB/s | 215 kB 00:00 2024-09-18T01:00:19.121 INFO:teuthology.orchestra.run.smithi164.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 823 kB/s | 172 kB 00:00 2024-09-18T01:00:19.147 INFO:teuthology.orchestra.run.smithi164.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 922 kB/s | 77 kB 00:00 2024-09-18T01:00:19.172 INFO:teuthology.orchestra.run.smithi164.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-18T01:00:19.214 INFO:teuthology.orchestra.run.smithi164.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 434 kB/s | 51 kB 00:00 2024-09-18T01:00:19.264 INFO:teuthology.orchestra.run.smithi164.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-18T01:00:19.289 INFO:teuthology.orchestra.run.smithi164.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-09-18T01:00:19.315 INFO:teuthology.orchestra.run.smithi164.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 440 kB/s | 43 kB 00:00 2024-09-18T01:00:19.349 INFO:teuthology.orchestra.run.smithi164.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-18T01:00:19.374 INFO:teuthology.orchestra.run.smithi164.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 210 kB/s | 11 kB 00:00 2024-09-18T01:00:19.399 INFO:teuthology.orchestra.run.smithi164.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 365 kB/s | 18 kB 00:00 2024-09-18T01:00:19.402 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-18T01:00:19.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-18T01:00:19.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-18T01:00:19.402 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T01:00:19.419 INFO:teuthology.orchestra.run.smithi098.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 349 kB/s | 253 kB 00:00 2024-09-18T01:00:19.424 INFO:teuthology.orchestra.run.smithi164.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-18T01:00:19.449 INFO:teuthology.orchestra.run.smithi164.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-18T01:00:19.475 INFO:teuthology.orchestra.run.smithi164.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-18T01:00:19.500 INFO:teuthology.orchestra.run.smithi164.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-18T01:00:19.525 INFO:teuthology.orchestra.run.smithi164.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-18T01:00:19.536 INFO:teuthology.orchestra.run.smithi098.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-18T01:00:19.551 INFO:teuthology.orchestra.run.smithi164.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-18T01:00:19.553 INFO:teuthology.orchestra.run.smithi098.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 228 kB/s | 106 kB 00:00 2024-09-18T01:00:19.576 INFO:teuthology.orchestra.run.smithi164.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-18T01:00:19.602 INFO:teuthology.orchestra.run.smithi164.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-18T01:00:19.627 INFO:teuthology.orchestra.run.smithi164.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-18T01:00:19.644 INFO:teuthology.orchestra.run.smithi164.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 531 kB/s | 36 kB 00:00 2024-09-18T01:00:19.662 INFO:teuthology.orchestra.run.smithi098.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 996 kB/s | 125 kB 00:00 2024-09-18T01:00:19.669 INFO:teuthology.orchestra.run.smithi164.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 807 kB/s | 20 kB 00:00 2024-09-18T01:00:19.694 INFO:teuthology.orchestra.run.smithi164.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-18T01:00:19.714 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:19.715 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:19.727 INFO:teuthology.orchestra.run.smithi164.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-18T01:00:19.731 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:19.732 INFO:teuthology.orchestra.run.smithi164.stdout:Total 11 MB/s | 30 MB 00:02 2024-09-18T01:00:19.762 INFO:teuthology.orchestra.run.smithi098.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.0 MB/s | 215 kB 00:00 2024-09-18T01:00:19.788 INFO:teuthology.orchestra.run.smithi098.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-18T01:00:19.838 INFO:teuthology.orchestra.run.smithi098.stdout:(13/37): python3-cryptography-36.0.1-4.el9.x86_ 1.1 MB/s | 1.2 MB 00:01 2024-09-18T01:00:19.863 INFO:teuthology.orchestra.run.smithi098.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 463 kB/s | 35 kB 00:00 2024-09-18T01:00:19.888 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:19.905 INFO:teuthology.orchestra.run.smithi098.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-18T01:00:19.927 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:19.927 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:19.930 INFO:teuthology.orchestra.run.smithi098.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-18T01:00:19.955 INFO:teuthology.orchestra.run.smithi098.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 657 kB/s | 77 kB 00:00 2024-09-18T01:00:19.981 INFO:teuthology.orchestra.run.smithi098.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-18T01:00:20.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-09-18T01:00:20.031 INFO:teuthology.orchestra.run.smithi098.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-18T01:00:20.064 INFO:teuthology.orchestra.run.smithi098.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 5.3 MB/s | 172 kB 00:00 2024-09-18T01:00:20.090 INFO:teuthology.orchestra.run.smithi098.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-09-18T01:00:20.115 INFO:teuthology.orchestra.run.smithi098.stdout:(22/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 324 kB/s | 43 kB 00:00 2024-09-18T01:00:20.140 INFO:teuthology.orchestra.run.smithi098.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-09-18T01:00:20.174 INFO:teuthology.orchestra.run.smithi098.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 901 kB/s | 23 kB 00:00 2024-09-18T01:00:20.199 INFO:teuthology.orchestra.run.smithi098.stdout:(25/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.6 MB/s | 358 kB 00:00 2024-09-18T01:00:20.224 INFO:teuthology.orchestra.run.smithi098.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 403 kB/s | 19 kB 00:00 2024-09-18T01:00:20.250 INFO:teuthology.orchestra.run.smithi098.stdout:(27/37): python3-jaraco-classes-3.2.1-5.el9.noa 131 kB/s | 18 kB 00:00 2024-09-18T01:00:20.275 INFO:teuthology.orchestra.run.smithi098.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-18T01:00:20.300 INFO:teuthology.orchestra.run.smithi098.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-09-18T01:00:20.326 INFO:teuthology.orchestra.run.smithi098.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-18T01:00:20.351 INFO:teuthology.orchestra.run.smithi098.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-18T01:00:20.372 INFO:teuthology.orchestra.run.smithi098.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.9 MB/s | 90 kB 00:00 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:20.386 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:00:20.388 INFO:teuthology.orchestra.run.smithi098.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 958 kB/s | 36 kB 00:00 2024-09-18T01:00:20.414 INFO:teuthology.orchestra.run.smithi098.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-18T01:00:20.431 INFO:teuthology.orchestra.run.smithi098.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-18T01:00:20.456 INFO:teuthology.orchestra.run.smithi098.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-18T01:00:20.481 INFO:teuthology.orchestra.run.smithi098.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-18T01:00:20.485 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:20.485 INFO:teuthology.orchestra.run.smithi098.stdout:Total 12 MB/s | 30 MB 00:02 2024-09-18T01:00:20.611 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:40 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:20.641 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:00:20.681 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:00:20.681 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:00:20.765 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:20.767 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:20.767 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:00:20.767 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-18T01:00:20.768 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T01:00:20.769 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T01:00:20.770 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout:Install 37 Packages 2024-09-18T01:00:20.771 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:20.772 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 30 M 2024-09-18T01:00:20.772 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 108 M 2024-09-18T01:00:20.772 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:00:21.143 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:00:21.143 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:00:21.442 INFO:teuthology.orchestra.run.smithi031.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 52 kB/s | 6.5 kB 00:00 2024-09-18T01:00:21.617 INFO:teuthology.orchestra.run.smithi031.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 5.0 MB/s | 1.5 MB 00:00 2024-09-18T01:00:21.684 INFO:teuthology.orchestra.run.smithi031.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.5 MB/s | 2.4 MB 00:00 2024-09-18T01:00:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:00:21.759 INFO:teuthology.orchestra.run.smithi031.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 3.2 MB/s | 249 kB 00:00 2024-09-18T01:00:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-18T01:00:21.862 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-18T01:00:21.891 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-18T01:00:21.935 INFO:teuthology.orchestra.run.smithi031.stdout:(5/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 9.6 MB/s | 4.7 MB 00:00 2024-09-18T01:00:21.965 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-18T01:00:22.018 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-18T01:00:22.051 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-18T01:00:22.147 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-18T01:00:22.168 INFO:teuthology.orchestra.run.smithi031.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-18T01:00:22.222 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-18T01:00:22.279 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-18T01:00:22.314 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-18T01:00:22.343 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-18T01:00:22.378 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-18T01:00:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-18T01:00:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-18T01:00:22.487 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-18T01:00:22.510 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:00:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-18T01:00:22.567 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-18T01:00:22.570 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-18T01:00:22.602 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-18T01:00:22.617 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-18T01:00:22.645 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-18T01:00:22.675 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-18T01:00:22.724 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-18T01:00:22.769 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-18T01:00:22.802 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-18T01:00:22.823 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-18T01:00:22.835 INFO:teuthology.orchestra.run.smithi031.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 235 kB/s | 253 kB 00:01 2024-09-18T01:00:22.860 INFO:teuthology.orchestra.run.smithi031.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-09-18T01:00:22.894 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-18T01:00:22.902 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-18T01:00:22.955 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-18T01:00:22.971 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-18T01:00:22.994 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-18T01:00:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-18T01:00:23.051 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-18T01:00:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-18T01:00:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-18T01:00:23.107 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-18T01:00:23.120 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-18T01:00:23.154 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-18T01:00:23.194 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-18T01:00:23.229 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-18T01:00:23.272 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-18T01:00:23.303 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-18T01:00:23.329 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-18T01:00:23.348 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-18T01:00:23.360 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-18T01:00:23.426 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-18T01:00:23.433 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-18T01:00:23.515 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-18T01:00:23.560 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-18T01:00:23.560 INFO:teuthology.orchestra.run.smithi031.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 186 kB/s | 135 kB 00:00 2024-09-18T01:00:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-18T01:00:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-18T01:00:23.644 INFO:teuthology.orchestra.run.smithi031.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 159 kB/s | 125 kB 00:00 2024-09-18T01:00:23.682 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-18T01:00:23.705 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-18T01:00:23.752 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-18T01:00:23.761 INFO:teuthology.orchestra.run.smithi031.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-18T01:00:23.801 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-18T01:00:23.819 INFO:teuthology.orchestra.run.smithi031.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-18T01:00:23.852 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-18T01:00:23.862 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-18T01:00:23.870 INFO:teuthology.orchestra.run.smithi031.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 839 kB/s | 35 kB 00:00 2024-09-18T01:00:23.882 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:00:23.907 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:00:23.907 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:23.907 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T01:00:23.907 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:00:23.907 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:00:23.907 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:23.912 INFO:teuthology.orchestra.run.smithi031.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-18T01:00:23.945 INFO:teuthology.orchestra.run.smithi031.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-18T01:00:23.979 INFO:teuthology.orchestra.run.smithi031.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-18T01:00:24.046 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-18T01:00:24.087 INFO:teuthology.orchestra.run.smithi031.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-18T01:00:24.095 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-18T01:00:24.158 INFO:teuthology.orchestra.run.smithi031.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 2.1 MB/s | 151 kB 00:00 2024-09-18T01:00:24.164 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-18T01:00:24.183 INFO:teuthology.orchestra.run.smithi031.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-18T01:00:24.216 INFO:teuthology.orchestra.run.smithi031.stdout:(20/37): python3-cryptography-36.0.1-4.el9.x86_ 550 kB/s | 1.2 MB 00:02 2024-09-18T01:00:24.252 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-18T01:00:24.267 INFO:teuthology.orchestra.run.smithi031.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-18T01:00:24.286 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-18T01:00:24.292 INFO:teuthology.orchestra.run.smithi031.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-18T01:00:24.317 INFO:teuthology.orchestra.run.smithi031.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-18T01:00:24.343 INFO:teuthology.orchestra.run.smithi031.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-18T01:00:24.368 INFO:teuthology.orchestra.run.smithi031.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-18T01:00:24.393 INFO:teuthology.orchestra.run.smithi031.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-18T01:00:24.403 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-18T01:00:24.418 INFO:teuthology.orchestra.run.smithi031.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-18T01:00:24.444 INFO:teuthology.orchestra.run.smithi031.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-18T01:00:24.469 INFO:teuthology.orchestra.run.smithi031.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-18T01:00:24.494 INFO:teuthology.orchestra.run.smithi031.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 657 kB/s | 16 kB 00:00 2024-09-18T01:00:24.545 INFO:teuthology.orchestra.run.smithi031.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-09-18T01:00:24.566 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-18T01:00:24.570 INFO:teuthology.orchestra.run.smithi031.stdout:(32/37): python3-urllib3-1.26.5-6.el9.noarch.rp 213 kB/s | 215 kB 00:01 2024-09-18T01:00:24.595 INFO:teuthology.orchestra.run.smithi031.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 732 kB/s | 36 kB 00:00 2024-09-18T01:00:24.596 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:00:24.624 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:00:24.624 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:24.624 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T01:00:24.624 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:00:24.624 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:00:24.624 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:24.629 INFO:teuthology.orchestra.run.smithi031.stdout:(34/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 671 kB/s | 90 kB 00:00 2024-09-18T01:00:24.662 INFO:teuthology.orchestra.run.smithi031.stdout:(35/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 7.1 MB/s | 427 kB 00:00 2024-09-18T01:00:24.687 INFO:teuthology.orchestra.run.smithi031.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 354 kB/s | 20 kB 00:00 2024-09-18T01:00:24.713 INFO:teuthology.orchestra.run.smithi031.stdout:(37/37): python3-webob-1.8.8-2.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-09-18T01:00:24.719 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:24.719 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.7 MB/s | 30 MB 00:03 2024-09-18T01:00:24.877 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:00:24.922 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:00:24.922 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:00:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:00:25.377 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:00:25.377 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:25.378 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T01:00:25.378 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:00:25.378 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:00:25.378 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:25.408 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:00:25.408 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:00:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:00:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:00:25.841 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:25.841 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T01:00:25.841 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:00:25.841 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:00:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:26.073 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:00:26.076 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:00:26.100 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:00:26.100 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:26.100 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T01:00:26.101 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:00:26.101 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:00:26.101 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:26.101 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:00:26.101 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:26.102 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T01:00:26.102 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:00:26.102 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:00:26.102 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:26.165 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:00:26.540 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:00:26.564 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:00:26.564 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:26.564 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T01:00:26.564 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:00:26.564 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:00:26.565 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:26.806 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:00:26.824 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:00:26.848 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:00:26.848 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:26.848 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T01:00:26.848 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:00:26.848 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:00:26.848 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-18T01:00:26.912 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:00:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-18T01:00:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-18T01:00:27.046 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-18T01:00:27.093 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-18T01:00:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-18T01:00:27.223 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-18T01:00:27.299 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-18T01:00:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-18T01:00:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-18T01:00:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-18T01:00:27.452 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-18T01:00:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-18T01:00:27.527 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-18T01:00:27.570 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-18T01:00:27.604 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-18T01:00:27.654 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-18T01:00:27.702 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-18T01:00:27.780 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-18T01:00:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-18T01:00:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:00:27.989 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-18T01:00:27.990 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-18T01:00:27.991 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-18T01:00:27.992 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-18T01:00:28.066 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-18T01:00:28.224 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-18T01:00:28.260 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-18T01:00:28.319 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-18T01:00:28.392 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-18T01:00:28.592 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-18T01:00:28.639 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-18T01:00:28.721 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-18T01:00:28.767 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-18T01:00:28.768 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-18T01:00:28.769 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-18T01:00:28.770 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-18T01:00:28.771 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-18T01:00:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-18T01:00:28.891 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-18T01:00:28.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-18T01:00:28.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T01:00:28.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T01:00:28.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T01:00:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T01:00:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T01:00:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T01:00:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T01:00:28.943 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:28.943 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:29.021 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-18T01:00:29.192 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-18T01:00:29.222 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:00:29.249 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-18T01:00:29.250 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:29.250 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T01:00:29.250 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:00:29.250 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T01:00:29.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:29.325 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-base 2024-09-18T01:00:29.759 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-18T01:00:29.760 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T01:00:29.761 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T01:00:29.762 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T01:00:29.763 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T01:00:29.763 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:29.763 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:29.907 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:52 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:29.996 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:00:30.037 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:30.038 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:00:30.038 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:30.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install cephadm 2024-09-18T01:00:30.137 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-base 2024-09-18T01:00:30.685 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:53 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:30.721 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:52 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:00:30.810 INFO:teuthology.orchestra.run.smithi098.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:00:30.810 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-18T01:00:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T01:00:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:00:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T01:00:30.811 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:30.817 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:00:30.818 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:30.819 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 765 k 2024-09-18T01:00:30.819 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 771 k 2024-09-18T01:00:30.819 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:30.852 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:30.853 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:00:30.853 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:30.917 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install cephadm 2024-09-18T01:00:31.078 INFO:teuthology.orchestra.run.smithi164.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-18T01:00:31.078 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:31.078 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-18T01:00:31.079 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:31.084 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:31.084 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:31.090 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:31.090 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:00:31.165 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:00:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:00:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-18T01:00:31.308 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:31.308 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T01:00:31.308 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:00:31.308 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T01:00:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:31.491 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:53 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:00:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-18T01:00:31.592 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:31.592 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T01:00:31.592 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:00:31.592 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T01:00:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:31.621 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:00:31.622 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:31.623 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:00:31.623 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:31.623 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 765 k 2024-09-18T01:00:31.623 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 771 k 2024-09-18T01:00:31.623 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:00:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:00:31.673 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:31.774 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi098.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:31.883 INFO:teuthology.orchestra.run.smithi098.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-18T01:00:31.884 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:00:31.889 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:00:31.894 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:00:31.895 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:00:31.977 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:00:32.138 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:32.401 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:32.401 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:32.401 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:32.401 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:32.511 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:32.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T01:00:32.861 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:33.105 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:33.250 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:56 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:33.329 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T01:00:33.385 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:33.385 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 145 k 2024-09-18T01:00:33.386 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 435 k 2024-09-18T01:00:33.387 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-18T01:00:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-18T01:00:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-18T01:00:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-18T01:00:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-18T01:00:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-18T01:00:33.568 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 798 kB/s | 145 kB 00:00 2024-09-18T01:00:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:Total 791 kB/s | 145 kB 00:00 2024-09-18T01:00:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:33.577 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:33.577 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:33.634 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:33.635 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:00:33.796 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:00:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:33.904 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:33.904 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T01:00:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:33.926 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:55 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:34.064 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:34.065 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:34.065 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repo Size 2024-09-18T01:00:34.065 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:34.065 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:00:34.065 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-18T01:00:34.065 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:34.066 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:00:34.066 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:34.066 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:00:34.066 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:34.066 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 145 k 2024-09-18T01:00:34.066 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 435 k 2024-09-18T01:00:34.066 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:00:34.281 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 676 kB/s | 145 kB 00:00 2024-09-18T01:00:34.281 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:34.281 INFO:teuthology.orchestra.run.smithi098.stdout:Total 672 kB/s | 145 kB 00:00 2024-09-18T01:00:34.282 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:00:34.289 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:00:34.289 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:00:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-18T01:00:34.336 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T01:00:34.337 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T01:00:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:34.339 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:34.345 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:00:34.346 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:00:34.491 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:00:34.548 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:34.573 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:34.573 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:34.573 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T01:00:34.573 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:34.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-09-18T01:00:34.861 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:34.862 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:34.862 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:34.862 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:34.862 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:34.862 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:35.105 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr 2024-09-18T01:00:35.305 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:55 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:35.396 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:00:35.442 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:35.443 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:00:35.443 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:35.496 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:00:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:35.496 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:35.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-09-18T01:00:35.696 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:58 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:35.708 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-mgr 2024-09-18T01:00:35.791 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:00:35.853 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:35.853 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:00:35.853 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:35.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T01:00:36.119 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:56 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 765 k 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 771 k 2024-09-18T01:00:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:00:36.302 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:58 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:36.394 INFO:teuthology.orchestra.run.smithi098.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:00:36.436 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:36.437 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:00:36.437 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:36.482 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:59 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:36.496 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T01:00:36.531 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-18T01:00:36.532 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:36.532 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-18T01:00:36.532 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:00:36.537 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:00:36.537 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:00:36.543 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:00:36.544 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:00:36.614 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-18T01:00:36.616 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout:Install 12 Packages 2024-09-18T01:00:36.617 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 8.7 M 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 104 M 2024-09-18T01:00:36.618 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:36.621 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:00:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 216 kB/s | 32 kB 00:00 2024-09-18T01:00:36.873 INFO:teuthology.orchestra.run.smithi164.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 95 kB/s | 16 kB 00:00 2024-09-18T01:00:37.007 INFO:teuthology.orchestra.run.smithi164.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-18T01:00:37.066 INFO:teuthology.orchestra.run.smithi164.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-09-18T01:00:37.066 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:00:59 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:37.085 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:37.124 INFO:teuthology.orchestra.run.smithi164.stdout:(5/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 8.4 MB/s | 3.6 MB 00:00 2024-09-18T01:00:37.174 INFO:teuthology.orchestra.run.smithi164.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 451 kB/s | 19 kB 00:00 2024-09-18T01:00:37.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:37.196 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-18T01:00:37.197 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:00:37.198 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi098.stdout:Install 12 Packages 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 8.7 M 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 104 M 2024-09-18T01:00:37.199 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:00:37.266 INFO:teuthology.orchestra.run.smithi164.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.7 MB/s | 548 kB 00:00 2024-09-18T01:00:37.309 INFO:teuthology.orchestra.run.smithi164.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-09-18T01:00:37.342 INFO:teuthology.orchestra.run.smithi164.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 930 kB/s | 31 kB 00:00 2024-09-18T01:00:37.367 INFO:teuthology.orchestra.run.smithi164.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.3 MB/s | 863 kB 00:00 2024-09-18T01:00:37.402 INFO:teuthology.orchestra.run.smithi164.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-18T01:00:37.526 INFO:teuthology.orchestra.run.smithi164.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.8 MB/s | 2.0 MB 00:00 2024-09-18T01:00:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:37.530 INFO:teuthology.orchestra.run.smithi164.stdout:Total 9.5 MB/s | 8.7 MB 00:00 2024-09-18T01:00:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:37.577 INFO:teuthology.orchestra.run.smithi098.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 217 kB/s | 32 kB 00:00 2024-09-18T01:00:37.579 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:37.596 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:37.597 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:37.602 INFO:teuthology.orchestra.run.smithi098.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 96 kB/s | 16 kB 00:00 2024-09-18T01:00:37.786 INFO:teuthology.orchestra.run.smithi098.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.4 MB/s | 269 kB 00:00 2024-09-18T01:00:37.805 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:37.805 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:00:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:00:37.816 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:37.816 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:00:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:37.817 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:37.817 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:37.844 INFO:teuthology.orchestra.run.smithi098.stdout:(4/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 8.6 MB/s | 3.6 MB 00:00 2024-09-18T01:00:37.886 INFO:teuthology.orchestra.run.smithi098.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-09-18T01:00:37.945 INFO:teuthology.orchestra.run.smithi098.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 377 kB/s | 19 kB 00:00 2024-09-18T01:00:38.012 INFO:teuthology.orchestra.run.smithi098.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.2 MB/s | 548 kB 00:00 2024-09-18T01:00:38.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T01:00:38.038 INFO:teuthology.orchestra.run.smithi098.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.4 MB/s | 863 kB 00:00 2024-09-18T01:00:38.062 INFO:teuthology.orchestra.run.smithi098.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-18T01:00:38.113 INFO:teuthology.orchestra.run.smithi098.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 408 kB/s | 31 kB 00:00 2024-09-18T01:00:38.138 INFO:teuthology.orchestra.run.smithi098.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.7 MB/s | 188 kB 00:00 2024-09-18T01:00:38.346 INFO:teuthology.orchestra.run.smithi098.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.1 MB/s | 2.0 MB 00:00 2024-09-18T01:00:38.350 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:38.350 INFO:teuthology.orchestra.run.smithi098.stdout:Total 7.5 MB/s | 8.7 MB 00:01 2024-09-18T01:00:38.357 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:00:38.398 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:00:38.417 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:00:38.417 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:00:38.435 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-18T01:00:38.479 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-18T01:00:38.525 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-18T01:00:38.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-18T01:00:38.612 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:00:38.613 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:00:38.647 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:58 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:38.648 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-18T01:00:38.774 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-18T01:00:38.785 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:38.786 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-18T01:00:38.786 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:38.786 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:00:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 145 k 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 435 k 2024-09-18T01:00:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:00:38.869 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-18T01:00:38.920 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-18T01:00:38.948 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:00:38.979 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-18T01:00:38.986 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 731 kB/s | 145 kB 00:00 2024-09-18T01:00:38.986 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:38.987 INFO:teuthology.orchestra.run.smithi031.stdout:Total 725 kB/s | 145 kB 00:00 2024-09-18T01:00:38.987 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:00:38.995 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:00:38.995 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:00:39.056 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:00:39.057 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:00:39.141 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:00:39.211 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:00:39.233 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-18T01:00:39.274 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-18T01:00:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:39.306 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:00:39.306 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T01:00:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:39.320 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-18T01:00:39.351 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-18T01:00:39.445 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-18T01:00:39.577 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-18T01:00:39.662 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-18T01:00:39.714 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-18T01:00:39.760 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:00:39.792 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-18T01:00:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-18T01:00:40.072 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:00:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-18T01:00:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:40.250 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:00:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:00:40.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:40.250 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:40.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-09-18T01:00:40.844 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-18T01:00:40.884 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:00:41.082 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:01 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:41.174 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:00:41.219 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:41.220 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:00:41.220 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:41.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-18T01:00:41.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-18T01:00:41.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-18T01:00:41.833 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-18T01:00:41.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-18T01:00:41.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-18T01:00:41.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-18T01:00:41.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T01:00:41.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T01:00:41.834 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:41.834 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:41.888 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:01 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-18T01:00:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-18T01:00:42.024 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-18T01:00:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-18T01:00:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:42.028 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 8.7 M 2024-09-18T01:00:42.028 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 104 M 2024-09-18T01:00:42.028 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:00:42.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T01:00:42.320 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-18T01:00:42.320 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:42.320 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:00:42.320 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-18T01:00:42.320 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:42.321 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:42.551 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T01:00:42.691 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:05 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:42.731 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 107 kB/s | 16 kB 00:00 2024-09-18T01:00:42.756 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): ceph-grafana-dashboards-19.1.1-330.gc2d 177 kB/s | 32 kB 00:00 2024-09-18T01:00:42.824 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:42.825 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-18T01:00:42.825 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:00:42.825 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T01:00:42.826 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T01:00:42.827 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T01:00:42.827 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T01:00:42.827 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:42.827 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:42.827 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-18T01:00:42.827 INFO:teuthology.orchestra.run.smithi164.stdout:Install 12 Packages 2024-09-18T01:00:42.827 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:42.828 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 38 M 2024-09-18T01:00:42.828 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 212 M 2024-09-18T01:00:42.828 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:42.890 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-18T01:00:42.948 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 9.6 MB/s | 3.6 MB 00:00 2024-09-18T01:00:42.982 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-09-18T01:00:43.026 INFO:teuthology.orchestra.run.smithi164.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-18T01:00:43.040 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 383 kB/s | 19 kB 00:00 2024-09-18T01:00:43.085 INFO:teuthology.orchestra.run.smithi164.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-18T01:00:43.127 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:05 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:43.127 INFO:teuthology.orchestra.run.smithi164.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-09-18T01:00:43.132 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.9 MB/s | 548 kB 00:00 2024-09-18T01:00:43.177 INFO:teuthology.orchestra.run.smithi164.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-18T01:00:43.183 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-18T01:00:43.208 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-18T01:00:43.236 INFO:teuthology.orchestra.run.smithi164.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 661 kB/s | 38 kB 00:00 2024-09-18T01:00:43.250 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-18T01:00:43.258 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:43.259 INFO:teuthology.orchestra.run.smithi098.stdout:====================================================================================== 2024-09-18T01:00:43.259 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout:====================================================================================== 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T01:00:43.260 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout:====================================================================================== 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout:Install 12 Packages 2024-09-18T01:00:43.261 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:43.262 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 38 M 2024-09-18T01:00:43.262 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 212 M 2024-09-18T01:00:43.262 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:00:43.345 INFO:teuthology.orchestra.run.smithi164.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-18T01:00:43.375 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.1 MB/s | 2.0 MB 00:00 2024-09-18T01:00:43.400 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.7 MB/s | 863 kB 00:00 2024-09-18T01:00:43.405 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:43.405 INFO:teuthology.orchestra.run.smithi031.stdout:Total 6.3 MB/s | 8.7 MB 00:01 2024-09-18T01:00:43.458 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:00:43.478 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:00:43.478 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:00:43.503 INFO:teuthology.orchestra.run.smithi164.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.1 MB/s | 3.0 MB 00:00 2024-09-18T01:00:43.529 INFO:teuthology.orchestra.run.smithi164.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-09-18T01:00:43.629 INFO:teuthology.orchestra.run.smithi164.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-09-18T01:00:43.654 INFO:teuthology.orchestra.run.smithi164.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-18T01:00:43.702 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:00:43.702 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:00:43.721 INFO:teuthology.orchestra.run.smithi164.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 10 MB/s | 4.9 MB 00:00 2024-09-18T01:00:44.005 INFO:teuthology.orchestra.run.smithi164.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-18T01:00:44.005 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:44.006 INFO:teuthology.orchestra.run.smithi164.stdout:Total 32 MB/s | 38 MB 00:01 2024-09-18T01:00:44.047 INFO:teuthology.orchestra.run.smithi098.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-18T01:00:44.189 INFO:teuthology.orchestra.run.smithi098.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 33 kB 00:00 2024-09-18T01:00:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:00:44.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-18T01:00:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-18T01:00:44.360 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:44.387 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:44.387 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-18T01:00:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-18T01:00:44.522 INFO:teuthology.orchestra.run.smithi098.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 205 kB/s | 191 kB 00:00 2024-09-18T01:00:44.545 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-18T01:00:44.606 INFO:teuthology.orchestra.run.smithi098.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 220 kB/s | 18 kB 00:00 2024-09-18T01:00:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-18T01:00:44.681 INFO:teuthology.orchestra.run.smithi098.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 512 kB/s | 38 kB 00:00 2024-09-18T01:00:44.778 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-18T01:00:44.815 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:44.815 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:00:44.827 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-18T01:00:44.863 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:00:44.895 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-18T01:00:44.957 INFO:teuthology.orchestra.run.smithi098.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.9 MB/s | 3.0 MB 00:00 2024-09-18T01:00:45.057 INFO:teuthology.orchestra.run.smithi098.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-18T01:00:45.282 INFO:teuthology.orchestra.run.smithi098.stdout:(8/12): libgfortran-11.5.0-2.el9.x86_64.rpm 473 kB/s | 801 kB 00:01 2024-09-18T01:00:45.449 INFO:teuthology.orchestra.run.smithi098.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.5 MB/s | 250 kB 00:00 2024-09-18T01:00:45.574 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:00:45.650 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T01:00:45.694 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:00:45.793 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:00:45.799 INFO:teuthology.orchestra.run.smithi098.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.4 MB/s | 4.9 MB 00:01 2024-09-18T01:00:45.837 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T01:00:46.007 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-18T01:00:46.022 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T01:00:46.037 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:00:46.057 INFO:teuthology.orchestra.run.smithi098.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.1 MB/s | 5.1 MB 00:01 2024-09-18T01:00:46.061 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T01:00:46.138 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T01:00:46.799 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T01:00:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-18T01:00:47.373 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-18T01:00:47.374 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-18T01:00:47.374 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-18T01:00:47.374 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-18T01:00:47.696 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-18T01:00:47.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:47.696 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:00:47.696 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-18T01:00:47.696 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:47.696 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:47.696 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:47.697 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:00:47.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T01:00:48.148 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:00:48.555 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:08 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:00:48.599 INFO:teuthology.orchestra.run.smithi098.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.2 MB/s | 16 MB 00:03 2024-09-18T01:00:48.600 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:48.601 INFO:teuthology.orchestra.run.smithi098.stdout:Total 7.2 MB/s | 38 MB 00:05 2024-09-18T01:00:48.694 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:00:48.695 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-18T01:00:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:00:48.695 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T01:00:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T01:00:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T01:00:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T01:00:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:48.697 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:00:48.697 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-18T01:00:48.697 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-18T01:00:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:00:48.698 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-09-18T01:00:48.698 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 212 M 2024-09-18T01:00:48.698 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:00:48.962 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:00:48.989 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:00:48.989 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:00:49.167 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T01:00:49.203 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:00:49.325 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-33 15 MB/s | 7.4 MB 00:00 2024-09-18T01:00:49.383 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 350 kB/s | 191 kB 00:00 2024-09-18T01:00:49.408 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-09-18T01:00:49.434 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:00:49.434 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:00:49.450 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-18T01:00:49.492 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 926 kB/s | 38 kB 00:00 2024-09-18T01:00:49.751 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 876 kB/s | 801 kB 00:00 2024-09-18T01:00:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-18T01:00:50.194 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:00:50.210 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.6 MB/s | 3.0 MB 00:00 2024-09-18T01:00:50.263 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T01:00:50.301 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.7 MB/s | 250 kB 00:00 2024-09-18T01:00:50.305 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:00:50.407 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:00:50.440 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T01:00:50.624 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T01:00:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.3 MB/s | 4.9 MB 00:01 2024-09-18T01:00:50.655 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T01:00:50.732 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T01:00:51.185 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.9 MB/s | 5.1 MB 00:01 2024-09-18T01:00:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T01:00:51.528 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T01:00:51.793 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:00:51.793 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-18T01:00:51.793 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:00:51.793 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:00:51.793 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T01:00:51.793 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T01:00:51.793 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T01:00:51.794 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T01:00:51.794 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T01:00:51.794 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T01:00:51.794 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:00:51.794 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T01:00:52.262 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:52.263 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:52.701 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-rook 2024-09-18T01:00:52.775 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:00:53.296 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:16 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:53.427 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:53.428 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:53.428 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:00:53.428 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T01:00:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout:Install 13 Packages 2024-09-18T01:00:53.430 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:53.431 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 2.2 M 2024-09-18T01:00:53.431 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 26 M 2024-09-18T01:00:53.431 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:53.815 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T01:00:53.839 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:00:53.997 INFO:teuthology.orchestra.run.smithi164.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 200 kB/s | 19 kB 00:00 2024-09-18T01:00:54.022 INFO:teuthology.orchestra.run.smithi164.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 212 kB/s | 26 kB 00:00 2024-09-18T01:00:54.081 INFO:teuthology.orchestra.run.smithi164.stdout:(3/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 274 kB/s | 49 kB 00:00 2024-09-18T01:00:54.106 INFO:teuthology.orchestra.run.smithi164.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-18T01:00:54.139 INFO:teuthology.orchestra.run.smithi164.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-18T01:00:54.165 INFO:teuthology.orchestra.run.smithi164.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 914 kB/s | 54 kB 00:00 2024-09-18T01:00:54.215 INFO:teuthology.orchestra.run.smithi164.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 429 kB/s | 32 kB 00:00 2024-09-18T01:00:54.240 INFO:teuthology.orchestra.run.smithi164.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 188 kB/s | 14 kB 00:00 2024-09-18T01:00:54.274 INFO:teuthology.orchestra.run.smithi164.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-18T01:00:54.300 INFO:teuthology.orchestra.run.smithi164.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-18T01:00:54.349 INFO:teuthology.orchestra.run.smithi164.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.0 MB/s | 90 kB 00:00 2024-09-18T01:00:54.375 INFO:teuthology.orchestra.run.smithi164.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-09-18T01:00:54.425 INFO:teuthology.orchestra.run.smithi164.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.5 MB/s | 1.0 MB 00:00 2024-09-18T01:00:54.427 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:54.428 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.2 MB/s | 2.2 MB 00:00 2024-09-18T01:00:54.492 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:54.507 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:54.507 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:54.568 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.8 MB/s | 16 MB 00:04 2024-09-18T01:00:54.569 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:54.569 INFO:teuthology.orchestra.run.smithi031.stdout:Total 6.5 MB/s | 38 MB 00:05 2024-09-18T01:00:54.598 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:54.601 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:00:54.943 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:00:54.971 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:00:54.971 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:00:55.136 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:00:55.322 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T01:00:55.376 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T01:00:55.440 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T01:00:55.441 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:00:55.441 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:00:55.477 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T01:00:55.527 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T01:00:55.637 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T01:00:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T01:00:55.799 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T01:00:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:00:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T01:00:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:00:56.423 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:00:56.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-18T01:00:56.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:00:56.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:00:56.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T01:00:56.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T01:00:56.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T01:00:56.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T01:00:56.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T01:00:56.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T01:00:56.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:00:56.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T01:00:56.464 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:00:56.502 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T01:00:56.518 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T01:00:56.576 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T01:00:56.610 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T01:00:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T01:00:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:00:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T01:00:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T01:00:56.825 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T01:00:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T01:00:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T01:00:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T01:00:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T01:00:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T01:00:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T01:00:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:56.856 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:00:57.203 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-mgr-rook 2024-09-18T01:00:57.527 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T01:00:57.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T01:00:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T01:00:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T01:00:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T01:00:57.529 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T01:00:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T01:00:57.658 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T01:00:57.792 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:19 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:00:57.867 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T01:00:57.867 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:57.867 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:00:57.867 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:00:57.867 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T01:00:57.867 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:57.868 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:00:57.925 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T01:00:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout:Install 13 Packages 2024-09-18T01:00:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:00:57.928 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 2.2 M 2024-09-18T01:00:57.928 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 26 M 2024-09-18T01:00:57.928 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:00:58.167 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T01:00:58.267 INFO:teuthology.orchestra.run.smithi098.stdout:(1/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 297 kB/s | 49 kB 00:00 2024-09-18T01:00:58.383 INFO:teuthology.orchestra.run.smithi098.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 91 kB/s | 26 kB 00:00 2024-09-18T01:00:58.409 INFO:teuthology.orchestra.run.smithi098.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 62 kB/s | 19 kB 00:00 2024-09-18T01:00:58.749 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:21 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:00:58.750 INFO:teuthology.orchestra.run.smithi098.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 432 kB/s | 159 kB 00:00 2024-09-18T01:00:58.859 INFO:teuthology.orchestra.run.smithi098.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 495 kB/s | 54 kB 00:00 2024-09-18T01:00:58.881 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T01:00:58.882 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:58.883 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:00:58.883 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:00:58.883 INFO:teuthology.orchestra.run.smithi164.stdout:Install 6 Packages 2024-09-18T01:00:58.883 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:00:58.883 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 7.0 M 2024-09-18T01:00:58.883 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 33 M 2024-09-18T01:00:58.883 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:00:58.884 INFO:teuthology.orchestra.run.smithi098.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 586 kB/s | 279 kB 00:00 2024-09-18T01:00:58.910 INFO:teuthology.orchestra.run.smithi098.stdout:(7/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 345 kB/s | 222 kB 00:00 2024-09-18T01:00:58.960 INFO:teuthology.orchestra.run.smithi098.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-18T01:00:58.985 INFO:teuthology.orchestra.run.smithi098.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-18T01:00:58.986 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:00:59.019 INFO:teuthology.orchestra.run.smithi098.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-18T01:00:59.044 INFO:teuthology.orchestra.run.smithi098.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-18T01:00:59.078 INFO:teuthology.orchestra.run.smithi098.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-18T01:00:59.144 INFO:teuthology.orchestra.run.smithi098.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.6 MB/s | 1.0 MB 00:00 2024-09-18T01:00:59.148 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:59.148 INFO:teuthology.orchestra.run.smithi098.stdout:Total 1.8 MB/s | 2.2 MB 00:01 2024-09-18T01:00:59.219 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:00:59.234 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:00:59.235 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:00:59.328 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:00:59.328 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:00:59.470 INFO:teuthology.orchestra.run.smithi164.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-18T01:00:59.570 INFO:teuthology.orchestra.run.smithi164.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 626 kB/s | 146 kB 00:00 2024-09-18T01:00:59.662 INFO:teuthology.orchestra.run.smithi164.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-18T01:00:59.696 INFO:teuthology.orchestra.run.smithi164.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.4 MB/s | 548 kB 00:00 2024-09-18T01:00:59.729 INFO:teuthology.orchestra.run.smithi164.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 363 kB/s | 58 kB 00:00 2024-09-18T01:00:59.763 INFO:teuthology.orchestra.run.smithi164.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 797 kB/s | 74 kB 00:00 2024-09-18T01:00:59.766 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:00:59.766 INFO:teuthology.orchestra.run.smithi164.stdout:Total 7.9 MB/s | 7.0 MB 00:00 2024-09-18T01:00:59.836 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:00:59.847 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:00:59.847 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:00:59.850 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:00:59.948 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:00:59.949 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:00.027 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T01:01:00.067 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T01:01:00.079 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T01:01:00.098 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:01:00.143 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T01:01:00.180 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T01:01:00.221 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T01:01:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T01:01:00.425 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T01:01:00.443 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T01:01:00.492 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T01:01:00.498 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T01:01:01.035 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T01:01:01.126 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T01:01:01.198 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T01:01:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T01:01:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:01:01.263 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T01:01:01.297 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T01:01:01.345 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T01:01:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:01:02.099 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:01:02.099 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-18T01:01:02.099 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T01:01:02.099 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T01:01:02.099 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T01:01:02.099 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T01:01:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:01:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-18T01:01:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T01:01:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T01:01:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T01:01:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T01:01:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T01:01:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T01:01:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T01:01:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T01:01:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T01:01:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T01:01:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T01:01:02.371 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:02.372 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:02.534 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T01:01:02.534 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:02.534 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:02.534 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:02.534 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T01:01:02.534 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T01:01:02.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T01:01:02.536 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:02.536 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:02.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-fuse 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T01:01:02.782 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T01:01:02.783 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T01:01:02.783 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T01:01:02.783 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T01:01:02.783 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T01:01:02.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T01:01:03.212 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:26 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:03.254 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T01:01:03.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:03.254 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:03.254 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:03.255 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:03.344 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:03.344 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:01:03.344 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:03.344 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:01:03.344 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-18T01:01:03.344 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-18T01:01:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T01:01:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:03.345 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:01:03.345 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:03.345 INFO:teuthology.orchestra.run.smithi164.stdout:Install 2 Packages 2024-09-18T01:01:03.345 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:03.345 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 977 k 2024-09-18T01:01:03.346 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 2.9 M 2024-09-18T01:01:03.346 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:01:03.408 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:25 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:03.540 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T01:01:03.541 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T01:01:03.542 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T01:01:03.542 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:03.542 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:03.542 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:03.542 INFO:teuthology.orchestra.run.smithi098.stdout:Install 6 Packages 2024-09-18T01:01:03.542 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:03.542 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 7.0 M 2024-09-18T01:01:03.543 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 33 M 2024-09-18T01:01:03.543 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:03.607 INFO:teuthology.orchestra.run.smithi164.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 548 kB/s | 80 kB 00:00 2024-09-18T01:01:03.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-09-18T01:01:03.740 INFO:teuthology.orchestra.run.smithi164.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.1 MB/s | 897 kB 00:00 2024-09-18T01:01:03.741 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:03.741 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.4 MB/s | 977 kB 00:00 2024-09-18T01:01:03.753 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:01:03.777 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:01:03.777 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:01:03.858 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:01:03.859 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:03.923 INFO:teuthology.orchestra.run.smithi098.stdout:(1/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 709 kB/s | 146 kB 00:00 2024-09-18T01:01:03.998 INFO:teuthology.orchestra.run.smithi098.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-18T01:01:04.023 INFO:teuthology.orchestra.run.smithi098.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-18T01:01:04.049 INFO:teuthology.orchestra.run.smithi098.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-18T01:01:04.064 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:04.205 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T01:01:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:24 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:04.282 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:01:04.357 INFO:teuthology.orchestra.run.smithi098.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 387 kB/s | 248 kB 00:00 2024-09-18T01:01:04.393 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:04.394 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:01:04.394 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:04.394 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:04.394 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-18T01:01:04.394 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T01:01:04.395 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:04.396 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:04.396 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:04.396 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-09-18T01:01:04.396 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:04.396 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-09-18T01:01:04.396 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-09-18T01:01:04.397 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:01:04.987 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-18T01:01:05.157 INFO:teuthology.orchestra.run.smithi098.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-18T01:01:05.159 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:05.159 INFO:teuthology.orchestra.run.smithi098.stdout:Total 4.3 MB/s | 7.0 MB 00:01 2024-09-18T01:01:05.230 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:05.242 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:05.242 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:05.292 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T01:01:05.292 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:05.292 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:05.292 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T01:01:05.292 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:05.293 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:05.341 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:05.342 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:05.365 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 298 kB/s | 49 kB 00:00 2024-09-18T01:01:05.491 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 88 kB/s | 26 kB 00:00 2024-09-18T01:01:05.516 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 61 kB/s | 19 kB 00:00 2024-09-18T01:01:05.583 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-volume 2024-09-18T01:01:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:05.833 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T01:01:05.875 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 414 kB/s | 159 kB 00:00 2024-09-18T01:01:05.884 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T01:01:05.916 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-09-18T01:01:05.958 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 775 kB/s | 32 kB 00:00 2024-09-18T01:01:05.984 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 563 kB/s | 14 kB 00:00 2024-09-18T01:01:06.009 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 400 kB/s | 54 kB 00:00 2024-09-18T01:01:06.043 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.7 MB/s | 220 kB 00:00 2024-09-18T01:01:06.068 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 506 kB/s | 279 kB 00:00 2024-09-18T01:01:06.093 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-18T01:01:06.160 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 979 kB/s | 90 kB 00:00 2024-09-18T01:01:06.186 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:29 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:06.219 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.9 MB/s | 1.0 MB 00:00 2024-09-18T01:01:06.222 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:06.222 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-09-18T01:01:06.296 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:06.313 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:06.313 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:06.324 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:06.325 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:06.325 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-18T01:01:06.326 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:06.326 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:01:06.326 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-18T01:01:06.326 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:06.326 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:01:06.327 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:06.327 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:01:06.327 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:06.327 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 285 k 2024-09-18T01:01:06.327 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 1.3 M 2024-09-18T01:01:06.328 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:01:06.407 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T01:01:06.414 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:06.415 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:06.496 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T01:01:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T01:01:06.562 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-18T01:01:06.563 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:06.563 INFO:teuthology.orchestra.run.smithi164.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-18T01:01:06.563 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:01:06.572 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:01:06.572 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:01:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:01:06.596 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:01:06.596 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:06.906 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:06.906 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:01:06.906 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T01:01:06.906 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:07.123 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T01:01:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T01:01:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T01:01:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T01:01:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T01:01:07.379 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:01:07.379 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-18T01:01:07.379 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T01:01:07.379 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T01:01:07.379 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T01:01:07.379 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T01:01:07.419 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T01:01:07.546 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T01:01:07.598 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T01:01:07.643 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T01:01:07.643 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:07.643 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:07.643 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:07.643 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T01:01:07.643 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T01:01:07.644 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T01:01:07.644 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T01:01:07.644 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T01:01:07.644 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:07.644 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:07.899 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-fuse 2024-09-18T01:01:07.936 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:07.936 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:07.936 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:07.936 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:07.937 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:07.937 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:08.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados-devel 2024-09-18T01:01:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T01:01:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T01:01:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T01:01:08.474 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:30 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T01:01:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:01:08.605 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:08.605 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:08.605 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:01:08.605 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout:Install 2 Packages 2024-09-18T01:01:08.606 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:08.607 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 977 k 2024-09-18T01:01:08.607 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 2.9 M 2024-09-18T01:01:08.607 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:08.795 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:31 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:08.826 INFO:teuthology.orchestra.run.smithi098.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 659 kB/s | 80 kB 00:00 2024-09-18T01:01:08.930 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:08.930 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:08.930 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-18T01:01:08.930 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:08.931 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:01:08.931 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-18T01:01:08.931 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:08.931 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:01:08.931 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:08.931 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:01:08.931 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:08.932 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 127 k 2024-09-18T01:01:08.932 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 456 k 2024-09-18T01:01:08.932 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:01:08.993 INFO:teuthology.orchestra.run.smithi098.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.0 MB/s | 897 kB 00:00 2024-09-18T01:01:08.993 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:08.994 INFO:teuthology.orchestra.run.smithi098.stdout:Total 2.5 MB/s | 977 kB 00:00 2024-09-18T01:01:09.005 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:09.027 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:09.027 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:09.108 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:09.108 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:09.152 INFO:teuthology.orchestra.run.smithi164.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 576 kB/s | 127 kB 00:00 2024-09-18T01:01:09.152 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:09.152 INFO:teuthology.orchestra.run.smithi164.stdout:Total 572 kB/s | 127 kB 00:00 2024-09-18T01:01:09.153 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:01:09.158 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:01:09.158 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:01:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:01:09.222 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:09.292 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T01:01:09.311 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T01:01:09.312 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T01:01:09.312 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T01:01:09.312 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T01:01:09.378 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:09.434 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T01:01:09.474 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:09.501 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:01:09.654 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T01:01:09.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:09.654 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T01:01:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T01:01:09.656 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:09.656 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:09.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T01:01:09.953 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:10.175 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:01:10.176 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-18T01:01:10.201 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:10.201 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:10.201 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:10.201 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:10.201 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:10.202 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:10.443 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T01:01:10.444 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:10.444 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:10.444 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T01:01:10.444 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:10.444 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:10.445 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs2 2024-09-18T01:01:10.537 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:30 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:10.675 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T01:01:10.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T01:01:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T01:01:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-09-18T01:01:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-09-18T01:01:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-09-18T01:01:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:10.714 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-volume 2024-09-18T01:01:11.037 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-18T01:01:11.047 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:11.062 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 719 kB/s | 146 kB 00:00 2024-09-18T01:01:11.112 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.3 MB/s | 58 kB 00:00 2024-09-18T01:01:11.140 INFO:teuthology.orchestra.run.smithi164.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:11.146 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-18T01:01:11.185 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:11.186 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:11.186 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:11.254 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-18T01:01:11.260 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs-devel 2024-09-18T01:01:11.314 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:33 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:11.451 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:11.452 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:11.452 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 285 k 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 1.3 M 2024-09-18T01:01:11.453 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:11.463 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-09-18T01:01:11.466 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:11.466 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.9 MB/s | 7.0 MB 00:00 2024-09-18T01:01:11.540 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:11.552 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:11.552 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:11.658 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:11.677 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:11.682 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-18T01:01:11.682 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:11.683 INFO:teuthology.orchestra.run.smithi098.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-18T01:01:11.683 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:11.691 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:11.691 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:11.716 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:11.716 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:11.843 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:11.934 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:11.975 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:11.975 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-18T01:01:11.975 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:11.975 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:01:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-18T01:01:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:11.976 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:01:11.976 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:11.976 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:01:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:11.976 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 32 k 2024-09-18T01:01:11.977 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 142 k 2024-09-18T01:01:11.977 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:01:11.977 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:12.001 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:12.017 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:12.017 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:01:12.017 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T01:01:12.017 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:12.123 INFO:teuthology.orchestra.run.smithi164.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 217 kB/s | 32 kB 00:00 2024-09-18T01:01:12.123 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:12.123 INFO:teuthology.orchestra.run.smithi164.stdout:Total 215 kB/s | 32 kB 00:00 2024-09-18T01:01:12.124 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:01:12.127 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:01:12.127 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:01:12.143 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:01:12.143 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T01:01:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T01:01:12.235 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:12.353 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:12.717 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T01:01:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T01:01:12.942 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T01:01:12.959 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:12.959 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:12.959 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:12.959 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:12.959 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:12.960 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:12.969 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:01:13.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:13.043 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:13.043 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:13.043 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:13.043 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:13.043 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:13.169 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados2 2024-09-18T01:01:13.365 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install librados-devel 2024-09-18T01:01:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-18T01:01:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-18T01:01:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T01:01:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T01:01:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T01:01:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T01:01:13.752 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:36 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:13.841 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:13.884 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:13.885 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:13.885 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:13.959 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:13.967 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librbd1 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T01:01:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:14.001 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:14.090 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repo Size 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:01:14.091 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:14.092 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 127 k 2024-09-18T01:01:14.092 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 456 k 2024-09-18T01:01:14.092 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:14.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-09-18T01:01:14.279 INFO:teuthology.orchestra.run.smithi098.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 676 kB/s | 127 kB 00:00 2024-09-18T01:01:14.280 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:14.280 INFO:teuthology.orchestra.run.smithi098.stdout:Total 670 kB/s | 127 kB 00:00 2024-09-18T01:01:14.281 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:14.286 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:14.286 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:14.350 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:14.350 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:14.551 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:14.640 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:14.683 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:14.684 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:14.685 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:14.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rados 2024-09-18T01:01:14.862 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:15.000 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:15.001 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:01:15.001 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:15.001 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-18T01:01:15.001 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-18T01:01:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T01:01:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:15.002 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:15.002 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:15.002 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-09-18T01:01:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:15.003 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 977 k 2024-09-18T01:01:15.003 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.9 M 2024-09-18T01:01:15.003 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:15.298 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 498 kB/s | 80 kB 00:00 2024-09-18T01:01:15.307 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:15.307 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:15.307 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:15.307 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:15.307 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:15.307 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:15.345 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:15.435 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:15.440 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 2.9 MB/s | 897 kB 00:00 2024-09-18T01:01:15.440 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:15.441 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.2 MB/s | 977 kB 00:00 2024-09-18T01:01:15.453 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:15.476 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:15.476 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:15.478 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:15.479 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:15.479 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:15.538 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install libcephfs2 2024-09-18T01:01:15.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rgw 2024-09-18T01:01:15.563 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:15.563 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T01:01:16.009 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:01:16.129 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:16.135 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:16.219 INFO:teuthology.orchestra.run.smithi098.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:16.225 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:16.264 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:16.265 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:16.265 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:16.268 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:16.269 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:16.269 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:16.324 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install libcephfs-devel 2024-09-18T01:01:16.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-cephfs 2024-09-18T01:01:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-18T01:01:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-18T01:01:16.907 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:38 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:16.920 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T01:01:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:16.987 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T01:01:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:16.987 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:17.008 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:17.040 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repo Size 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:01:17.041 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:17.042 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 32 k 2024-09-18T01:01:17.042 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 142 k 2024-09-18T01:01:17.042 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:17.051 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:17.052 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:17.052 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:17.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rbd 2024-09-18T01:01:17.192 INFO:teuthology.orchestra.run.smithi098.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 211 kB/s | 32 kB 00:00 2024-09-18T01:01:17.193 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:17.193 INFO:teuthology.orchestra.run.smithi098.stdout:Total 209 kB/s | 32 kB 00:00 2024-09-18T01:01:17.193 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:17.197 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:17.197 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:17.214 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:17.214 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:17.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-volume 2024-09-18T01:01:17.308 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:17.407 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:17.727 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:40 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:17.783 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:17.817 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:17.860 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:17.861 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:17.861 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:17.899 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:17.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-fuse 2024-09-18T01:01:18.016 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:18.017 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:18.017 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:18.017 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:18.017 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:18.017 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:18.036 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:18.037 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:01:18.038 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:18.038 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 285 k 2024-09-18T01:01:18.038 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.3 M 2024-09-18T01:01:18.038 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:18.183 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install librados2 2024-09-18T01:01:18.252 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.3 MB/s | 285 kB 00:00 2024-09-18T01:01:18.253 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:18.253 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-18T01:01:18.253 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:18.262 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:18.263 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:18.289 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:18.289 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:18.520 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:41 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:18.521 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:18.605 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:01:18.605 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T01:01:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:18.654 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:18.654 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:18.654 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 85 k 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 231 k 2024-09-18T01:01:18.655 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:01:18.760 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:40 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:18.848 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:18.851 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 436 kB/s | 85 kB 00:00 2024-09-18T01:01:18.852 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:18.852 INFO:teuthology.orchestra.run.smithi164.stdout:Total 433 kB/s | 85 kB 00:00 2024-09-18T01:01:18.853 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:01:18.859 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:01:18.860 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:01:18.891 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:18.892 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:18.892 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:18.922 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:01:18.922 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:18.960 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install librbd1 2024-09-18T01:01:19.063 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:19.166 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:19.539 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:41 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:19.573 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-18T01:01:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:19.594 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-18T01:01:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:19.594 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:19.629 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:19.672 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:19.673 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:19.673 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:19.733 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install python3-rados 2024-09-18T01:01:19.796 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:19.796 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:19.797 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:19.797 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:19.797 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:19.797 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:19.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-09-18T01:01:20.074 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-mirror 2024-09-18T01:01:20.310 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:20.398 INFO:teuthology.orchestra.run.smithi098.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:20.441 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:20.442 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:20.442 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:20.503 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install python3-rgw 2024-09-18T01:01:20.507 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:40 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:20.646 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:20.647 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:20.648 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:01:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:20.648 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-09-18T01:01:20.648 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 456 k 2024-09-18T01:01:20.648 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:20.679 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:43 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:20.818 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:20.819 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:20.819 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-18T01:01:20.819 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:20.819 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 3.1 M 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 13 M 2024-09-18T01:01:20.820 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:01:20.840 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 658 kB/s | 127 kB 00:00 2024-09-18T01:01:20.841 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:20.841 INFO:teuthology.orchestra.run.smithi031.stdout:Total 653 kB/s | 127 kB 00:00 2024-09-18T01:01:20.841 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:20.847 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:20.847 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:20.915 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:20.915 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:21.072 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:21.085 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:43 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:21.165 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.0 MB/s | 3.1 MB 00:00 2024-09-18T01:01:21.166 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:21.166 INFO:teuthology.orchestra.run.smithi164.stdout:Total 9.0 MB/s | 3.1 MB 00:00 2024-09-18T01:01:21.166 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:01:21.175 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:01:21.175 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:01:21.176 INFO:teuthology.orchestra.run.smithi098.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:21.219 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:21.220 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:21.220 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:21.256 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:01:21.257 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:21.281 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install python3-cephfs 2024-09-18T01:01:21.628 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:21.783 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:01:21.783 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T01:01:21.783 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:01:21.783 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:01:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:21.856 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:43 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:21.920 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:21.921 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:21.945 INFO:teuthology.orchestra.run.smithi098.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:21.988 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:21.989 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:21.989 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:22.047 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install python3-rbd 2024-09-18T01:01:22.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-09-18T01:01:22.632 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:44 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:22.721 INFO:teuthology.orchestra.run.smithi098.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:22.764 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:22.765 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:22.765 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:22.765 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:22.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install rbd-fuse 2024-09-18T01:01:22.857 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:22.903 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:22.904 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:22.904 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:22.951 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:22.951 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:22.951 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:22.951 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:22.952 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:22.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-09-18T01:01:23.184 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-nbd 2024-09-18T01:01:23.397 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:45 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:23.528 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:01:23.529 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:23.530 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 85 k 2024-09-18T01:01:23.530 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 231 k 2024-09-18T01:01:23.530 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:23.585 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:43 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:23.721 INFO:teuthology.orchestra.run.smithi098.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 448 kB/s | 85 kB 00:00 2024-09-18T01:01:23.721 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:23.722 INFO:teuthology.orchestra.run.smithi098.stdout:Total 445 kB/s | 85 kB 00:00 2024-09-18T01:01:23.722 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:23.723 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:23.723 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:01:23.724 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:23.725 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 32 k 2024-09-18T01:01:23.725 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 142 k 2024-09-18T01:01:23.725 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:23.728 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:23.728 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:23.781 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:46 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:23.792 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:23.793 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:23.876 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 210 kB/s | 32 kB 00:00 2024-09-18T01:01:23.877 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:23.877 INFO:teuthology.orchestra.run.smithi031.stdout:Total 208 kB/s | 32 kB 00:00 2024-09-18T01:01:23.877 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:23.881 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:23.881 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:23.897 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:23.898 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:23.915 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-18T01:01:23.916 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:23.917 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 170 k 2024-09-18T01:01:23.917 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 494 k 2024-09-18T01:01:23.917 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-18T01:01:23.947 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:24.051 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.118 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.128 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 809 kB/s | 170 kB 00:00 2024-09-18T01:01:24.128 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:24.128 INFO:teuthology.orchestra.run.smithi164.stdout:Total 803 kB/s | 170 kB 00:00 2024-09-18T01:01:24.129 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-18T01:01:24.136 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-18T01:01:24.136 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-18T01:01:24.200 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-18T01:01:24.201 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-18T01:01:24.356 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-18T01:01:24.433 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.450 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.692 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.693 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:24.693 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:24.693 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:24.693 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:24.693 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:24.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.740 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:24.740 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:24.740 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:24.740 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:24.740 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:24.917 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:24.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-09-18T01:01:24.941 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install rbd-mirror 2024-09-18T01:01:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:25.222 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-18T01:01:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:25.222 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:25.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-nbd 2024-09-18T01:01:25.542 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:45 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:25.563 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:47 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:25.637 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:25.682 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:25.683 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:25.683 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:25.703 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:25.704 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:25.704 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repo Size 2024-09-18T01:01:25.704 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:25.704 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:25.704 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-18T01:01:25.704 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:25.704 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:25.705 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:25.705 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:01:25.705 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:25.705 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 3.1 M 2024-09-18T01:01:25.705 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 13 M 2024-09-18T01:01:25.706 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:25.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-09-18T01:01:26.042 INFO:teuthology.orchestra.run.smithi098.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.2 MB/s | 3.1 MB 00:00 2024-09-18T01:01:26.042 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:26.043 INFO:teuthology.orchestra.run.smithi098.stdout:Total 9.2 MB/s | 3.1 MB 00:00 2024-09-18T01:01:26.043 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:26.052 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:26.052 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:26.081 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:49 ago on Wed 18 Sep 2024 12:59:37 AM UTC. 2024-09-18T01:01:26.135 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:26.135 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:26.172 INFO:teuthology.orchestra.run.smithi164.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:26.216 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-18T01:01:26.217 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-18T01:01:26.217 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-18T01:01:26.308 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:26.382 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:46 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:26.474 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:26.520 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:26.521 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:26.521 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:26.565 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:26.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-09-18T01:01:26.617 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:26.644 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:26.644 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:01:26.644 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T01:01:26.644 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:01:26.644 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:01:26.644 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:27.212 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:47 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:27.304 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:27.350 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:27.351 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:27.351 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:27.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-09-18T01:01:27.790 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:27.790 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:27.791 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:27.791 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:27.791 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:27.791 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:28.012 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install rbd-nbd 2024-09-18T01:01:28.050 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:48 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:28.142 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:28.188 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:28.189 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:28.189 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:28.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-09-18T01:01:28.602 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:50 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:28.735 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:Install 1 Package 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 170 k 2024-09-18T01:01:28.736 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 494 k 2024-09-18T01:01:28.737 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-18T01:01:28.871 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:48 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:28.946 INFO:teuthology.orchestra.run.smithi098.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 815 kB/s | 170 kB 00:00 2024-09-18T01:01:28.947 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:28.947 INFO:teuthology.orchestra.run.smithi098.stdout:Total 809 kB/s | 170 kB 00:00 2024-09-18T01:01:28.947 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-18T01:01:28.954 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-18T01:01:28.955 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-18T01:01:28.963 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:29.008 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:29.009 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:29.010 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:29.019 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-18T01:01:29.019 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-18T01:01:29.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-09-18T01:01:29.178 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-18T01:01:29.311 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:29.694 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:49 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:29.719 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:29.785 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:29.831 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:29.832 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:29.832 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:29.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-09-18T01:01:29.964 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:29.964 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:29.964 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-18T01:01:29.964 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:29.964 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:29.965 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:30.207 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install rbd-nbd 2024-09-18T01:01:30.528 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:50 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:30.670 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 85 k 2024-09-18T01:01:30.671 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 231 k 2024-09-18T01:01:30.672 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:30.802 INFO:teuthology.orchestra.run.smithi098.stdout:Last metadata expiration check: 0:01:52 ago on Wed 18 Sep 2024 12:59:38 AM UTC. 2024-09-18T01:01:30.871 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 428 kB/s | 85 kB 00:00 2024-09-18T01:01:30.872 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:30.873 INFO:teuthology.orchestra.run.smithi031.stdout:Total 424 kB/s | 85 kB 00:00 2024-09-18T01:01:30.873 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:30.880 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:30.880 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:30.893 INFO:teuthology.orchestra.run.smithi098.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:30.938 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-18T01:01:30.939 INFO:teuthology.orchestra.run.smithi098.stdout:Nothing to do. 2024-09-18T01:01:30.939 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-18T01:01:30.948 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:30.948 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:30.999 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:32.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-09-18T01:01:32.800 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:52 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:32.939 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:32.940 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:32.940 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-18T01:01:32.940 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.1 M 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 13 M 2024-09-18T01:01:32.941 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:33.274 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.3 MB/s | 3.1 MB 00:00 2024-09-18T01:01:33.275 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:33.275 INFO:teuthology.orchestra.run.smithi031.stdout:Total 9.3 MB/s | 3.1 MB 00:00 2024-09-18T01:01:33.275 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:33.285 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:33.285 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:33.373 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:33.373 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:33.850 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:33.908 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:33.936 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T01:01:33.936 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T01:01:33.936 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:01:33.936 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T01:01:33.936 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:35.055 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:35.056 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:35.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-09-18T01:01:35.953 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:55 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:36.094 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:36.094 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:36.094 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 170 k 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 494 k 2024-09-18T01:01:36.095 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-18T01:01:36.310 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 794 kB/s | 170 kB 00:00 2024-09-18T01:01:36.311 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-18T01:01:36.311 INFO:teuthology.orchestra.run.smithi031.stdout:Total 788 kB/s | 170 kB 00:00 2024-09-18T01:01:36.311 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-18T01:01:36.319 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-18T01:01:36.320 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-18T01:01:36.387 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-18T01:01:36.387 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-18T01:01:36.548 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-18T01:01:36.655 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-18T01:01:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:37.380 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-18T01:01:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-18T01:01:37.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:37.380 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:37.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-09-18T01:01:38.252 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:58 ago on Wed 18 Sep 2024 12:59:40 AM UTC. 2024-09-18T01:01:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-18T01:01:38.391 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-18T01:01:38.392 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-18T01:01:38.392 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-18T01:01:38.474 DEBUG:teuthology.parallel:result is None 2024-09-18T01:01:38.475 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T01:01:38.624 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T01:01:38.646 INFO:teuthology.orchestra.run.smithi031.stdout:19.1.1-330.gc2ddf972.el9 2024-09-18T01:01:38.647 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-18T01:01:38.647 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-18T01:01:38.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T01:01:38.795 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T01:01:38.821 INFO:teuthology.orchestra.run.smithi098.stdout:19.1.1-330.gc2ddf972.el9 2024-09-18T01:01:38.821 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-18T01:01:38.821 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-18T01:01:38.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-18T01:01:38.965 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T01:01:38.990 INFO:teuthology.orchestra.run.smithi164.stdout:19.1.1-330.gc2ddf972.el9 2024-09-18T01:01:38.991 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-18T01:01:38.991 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-18T01:01:38.993 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T01:01:38.993 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:38.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T01:01:39.020 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:01:39.020 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T01:01:39.051 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T01:01:39.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T01:01:39.080 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T01:01:39.080 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:39.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T01:01:39.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T01:01:39.173 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:01:39.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T01:01:39.201 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T01:01:39.268 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T01:01:39.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T01:01:39.298 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T01:01:39.367 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T01:01:39.368 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:39.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T01:01:39.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T01:01:39.461 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:01:39.461 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T01:01:39.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T01:01:39.560 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T01:01:39.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T01:01:39.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T01:01:39.661 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T01:01:39.661 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:39.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T01:01:39.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T01:01:39.754 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:01:39.754 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T01:01:39.784 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T01:01:39.854 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T01:01:39.854 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T01:01:39.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T01:01:39.954 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T01:01:40.033 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T01:01:40.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-09-18T01:01:40.035 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /var/log/ceph 2024-09-18T01:01:40.038 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-09-18T01:01:40.068 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T01:01:40.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T01:01:40.102 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T01:01:40.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T01:01:40.137 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T01:01:40.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T01:01:40.168 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T01:01:40.176 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T01:01:40.215 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T01:01:40.216 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T01:01:40.216 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909691', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBISZwA441xiMZLm497ZwZmlSvie/fcJ2cbYVRqn9xFwMiYDEIoXm5JaPJ92n/si9xlyr7i0+vFjiam3tDs4pvNY=', 'smithi098.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBkQo1omx3/jkcfnR/ZRolbFMe+tksQMdhFSiGOehaiffUP476vaXBc9E8oyR434mnC6T9iF/17QImCUj6fB57k=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMQFH3eaEC9zyNXB5ioUQB3PxZOiB+5M328saKGTTH0wW6rCaPvqCJnpf4v8b0lcyeObJBWJqGmtuioz+TQ9dTM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T01:01:40.216 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T01:01:40.234 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T01:01:40.246 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T01:01:40.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T01:01:40.291 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T01:01:40.303 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T01:01:40.334 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:40.334 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T01:01:40.371 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T01:01:40.371 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-18T01:01:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T01:01:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:40.426 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-18T01:01:40.426 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:40.426 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:40.427 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-18 01:01:34.196688291 +0000 2024-09-18T01:01:40.427 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-18 00:58:52.399724208 +0000 2024-09-18T01:01:40.427 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-18 00:58:52.399724208 +0000 2024-09-18T01:01:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-18 00:58:52.399724208 +0000 2024-09-18T01:01:40.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T01:01:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-18T01:01:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-18T01:01:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000254976 s, 2.0 MB/s 2024-09-18T01:01:40.492 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T01:01:40.549 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-18T01:01:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T01:01:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:40.605 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-18T01:01:40.606 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:40.606 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:40.606 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-18 01:01:34.196688291 +0000 2024-09-18T01:01:40.606 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-18 00:58:52.747719186 +0000 2024-09-18T01:01:40.606 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-18 00:58:52.747719186 +0000 2024-09-18T01:01:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-18 00:58:52.747719186 +0000 2024-09-18T01:01:40.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T01:01:40.668 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-18T01:01:40.669 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-18T01:01:40.669 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000211697 s, 2.4 MB/s 2024-09-18T01:01:40.673 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T01:01:40.730 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-18T01:01:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-18 01:01:34.197688276 +0000 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-18 00:58:53.081714366 +0000 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-18 00:58:53.081714366 +0000 2024-09-18T01:01:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-18 00:58:53.081714366 +0000 2024-09-18T01:01:40.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T01:01:40.851 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-18T01:01:40.851 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-18T01:01:40.851 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.0002001 s, 2.6 MB/s 2024-09-18T01:01:40.852 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T01:01:40.910 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-18T01:01:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-18 01:01:34.197688276 +0000 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-18 00:58:53.414709560 +0000 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-18 00:58:53.414709560 +0000 2024-09-18T01:01:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-18 00:58:53.414709560 +0000 2024-09-18T01:01:40.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T01:01:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-18T01:01:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-18T01:01:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000200302 s, 2.6 MB/s 2024-09-18T01:01:41.032 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T01:01:41.089 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:41.089 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:01:41.089 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T01:01:41.108 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T01:01:41.108 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-09-18T01:01:41.167 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T01:01:41.167 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:41.167 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-18T01:01:41.167 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:41.167 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:41.167 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-18 01:01:26.912060539 +0000 2024-09-18T01:01:41.168 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-18 00:58:50.315936125 +0000 2024-09-18T01:01:41.168 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-18 00:58:50.315936125 +0000 2024-09-18T01:01:41.168 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-18 00:58:50.315936125 +0000 2024-09-18T01:01:41.168 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T01:01:41.234 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-18T01:01:41.235 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-18T01:01:41.235 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000222036 s, 2.3 MB/s 2024-09-18T01:01:41.236 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T01:01:41.294 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-18 01:01:26.912060539 +0000 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-18 00:58:50.658931836 +0000 2024-09-18T01:01:41.352 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-18 00:58:50.658931836 +0000 2024-09-18T01:01:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-18 00:58:50.658931836 +0000 2024-09-18T01:01:41.353 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T01:01:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-18T01:01:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-18T01:01:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000275362 s, 1.9 MB/s 2024-09-18T01:01:41.419 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T01:01:41.476 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-18 01:01:26.912060539 +0000 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-18 00:58:50.999927572 +0000 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-18 00:58:50.999927572 +0000 2024-09-18T01:01:41.534 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-18 00:58:50.999927572 +0000 2024-09-18T01:01:41.535 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T01:01:41.599 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-18T01:01:41.599 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-18T01:01:41.599 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000208871 s, 2.5 MB/s 2024-09-18T01:01:41.600 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T01:01:41.657 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-09-18T01:01:41.712 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-18 01:01:26.913060526 +0000 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-18 00:58:51.333923394 +0000 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-18 00:58:51.333923394 +0000 2024-09-18T01:01:41.713 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-18 00:58:51.333923394 +0000 2024-09-18T01:01:41.713 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T01:01:41.777 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-18T01:01:41.777 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-18T01:01:41.778 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000286037 s, 1.8 MB/s 2024-09-18T01:01:41.779 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T01:01:41.835 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:41.836 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-18T01:01:41.836 INFO:tasks.ceph:Generating config... 2024-09-18T01:01:41.837 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T01:01:41.837 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-18T01:01:41.837 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T01:01:41.837 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T01:01:41.837 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T01:01:41.837 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T01:01:41.838 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T01:01:41.838 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T01:01:41.838 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T01:01:41.839 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T01:01:41.839 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-18T01:01:41.839 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T01:01:41.839 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-18T01:01:41.839 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T01:01:41.839 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T01:01:41.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T01:01:41.889 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T01:01:41.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T01:01:41.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T01:01:41.963 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]'), ('mon.b', '172.21.15.98')] 2024-09-18T01:01:41.963 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.98', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-18T01:01:41.964 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:41.964 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T01:01:42.020 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --add b 172.21.15.98 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:42.108 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:42.108 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 62b46b83-8ef2-466f-b28b-2d97f265422e 2024-09-18T01:01:42.108 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-09-18T01:01:42.108 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-09-18T01:01:42.108 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 62b46b83-8ef2-466f-b28b-2d97f265422e 2024-09-18T01:01:42.108 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-09-18T01:01:42.107848+0000 2024-09-18T01:01:42.108 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-18T01:01:42.107848+0000 2024-09-18T01:01:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 17 (quincy) 2024-09-18T01:01:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-09-18T01:01:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-18T01:01:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-09-18T01:01:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2024-09-18T01:01:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T01:01:42.110 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T01:01:42.126 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 62b46b83-8ef2-466f-b28b-2d97f265422e... 2024-09-18T01:01:42.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T01:01:42.168 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T01:01:42.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.210 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T01:01:42.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-09-18T01:01:42.212 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.98" 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: mon client directed command retry = 5 2024-09-18T01:01:42.213 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on pool no app = False 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: ms inject socket failures = 5000 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 62b46b83-8ef2-466f-b28b-2d97f265422e 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.214 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.215 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression algorithm = lz4 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression mode = aggressive 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-09-18T01:01:42.216 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: osd max markdown count = 1000 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.217 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-18T01:01:42.218 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T01:01:42.219 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T01:01:42.220 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi098.stdout:[global] 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi098.stdout: chdir = "" 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi098.stdout: auth supported = cephx 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.221 INFO:teuthology.orchestra.run.smithi098.stdout: filestore xattr use omap = true 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: mon clock drift allowed = 1.000 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: osd crush chooseleaf type = 0 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: auth debug = true 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on old message = true 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on bug = true 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: mon pg warn max object skew = 0 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.222 INFO:teuthology.orchestra.run.smithi098.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default size = 2 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow primary affinity = true 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow pg remap = true 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on legacy crush tunables = false 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on crush straw calc version zero = false 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on no sortbitwise = false 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on osd down out interval zero = false 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on too few osds = false 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T01:01:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: mon_allow_pool_size_one = true 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: osd default data pool replay window = 5 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: mon allow pool delete = true 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: mon cluster log file level = debug 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: debug asserts on shutdown = true 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: mon health detail to clog = false 2024-09-18T01:01:42.224 INFO:teuthology.orchestra.run.smithi098.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.98" 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: mon client directed command retry = 5 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on pool no app = False 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: ms inject socket failures = 5000 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: fsid = 62b46b83-8ef2-466f-b28b-2d97f265422e 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout:[osd] 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: osd journal size = 100 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub load threshold = 5.0 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub max interval = 600 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: osd mclock profile = high_recovery_ops 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: osd recover clone overlap = true 2024-09-18T01:01:42.225 INFO:teuthology.orchestra.run.smithi098.stdout: osd recovery max chunk = 1048576 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug shutdown = true 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug op order = true 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug verify stray on activate = true 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug trim objects = true 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd open classes on start = true 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug pg log writeout = true 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd deep scrub update digest min age = 30 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: osd map max advance = 10 2024-09-18T01:01:42.226 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: journal zero on create = true 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: filestore ondisk finisher threads = 3 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: filestore apply finisher threads = 3 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: bdev debug aio = true 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug misdirected ops = true 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore block size = 96636764160 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore compression algorithm = lz4 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore compression mode = aggressive 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore fsck on mount = True 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluefs = 1/20 2024-09-18T01:01:42.227 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluestore = 1/20 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: debug osd = 20 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: debug rocksdb = 4/10 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd full ratio = 0.9 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd nearfull ratio = 0.8 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: osd blocked scrub grace period = 3600 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: osd failsafe full ratio = 0.95 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: osd max markdown count = 1000 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: osd objectstore = bluestore 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr] 2024-09-18T01:01:42.228 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: debug mgr = 20 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: mgr/telemetry/nag = false 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout:[mon] 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: debug paxos = 20 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-09-18T01:01:42.229 INFO:teuthology.orchestra.run.smithi098.stdout: mon data avail warn = 5 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: mon mgr mkfs grace = 240 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd reporter subtree level = osd 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd prime pg temp = true 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T01:01:42.230 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: # 1m isn't quite enough 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: mon_down_mkfs_grace = 2m 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_filestore_osds = false 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout:[client] 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: rgw cache enabled = true 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable ops log = true 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable usage log = true 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T01:01:42.231 INFO:teuthology.orchestra.run.smithi098.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T01:01:42.232 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.a] 2024-09-18T01:01:42.232 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.c] 2024-09-18T01:01:42.232 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.b] 2024-09-18T01:01:42.232 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-09-18T01:01:42.232 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-09-18T01:01:42.233 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-09-18T01:01:42.234 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-09-18T01:01:42.235 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.98" 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: mon client directed command retry = 5 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on pool no app = False 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: ms inject socket failures = 5000 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 62b46b83-8ef2-466f-b28b-2d97f265422e 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-09-18T01:01:42.236 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-09-18T01:01:42.237 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-09-18T01:01:42.238 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression algorithm = lz4 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression mode = aggressive 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: osd blocked scrub grace period = 3600 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-09-18T01:01:42.239 INFO:teuthology.orchestra.run.smithi164.stdout: osd max markdown count = 1000 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.240 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.241 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-09-18T01:01:42.242 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-09-18T01:01:42.243 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-09-18T01:01:42.243 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-09-18T01:01:42.243 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T01:01:42.243 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T01:01:42.243 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-09-18T01:01:42.243 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-09-18T01:01:42.243 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-09-18T01:01:42.245 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T01:01:42.245 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T01:01:42.340 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T01:01:42.341 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:42.341 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T01:01:42.356 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:42.357 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T01:01:42.412 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:01:42.413 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:42.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T01:01:42.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T01:01:42.488 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:01:42.488 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:42.545 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:01:42.545 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:01:42.545 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T01:01:42.545 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T01:01:42.579 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:01:42.579 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:42.634 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-09-18T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T01:01:42.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T01:01:42.676 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T01:01:42.676 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:42.734 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T01:01:42.734 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T01:01:42.734 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T01:01:42.790 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T01:01:42.793 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T01:01:42.793 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T01:01:42.793 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T01:01:42.849 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T01:01:42.863 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T01:01:42.864 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-18T01:01:42.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T01:01:42.890 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:42.890 INFO:tasks.ceph:role: osd.0 2024-09-18T01:01:42.890 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:01:42.890 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:01:42.963 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:01:42.964 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:01:42.964 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:01:43.479 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-18T01:01:43.481 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-18T01:01:43.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T01:01:43.614 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T01:01:43.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T01:01:43.720 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:43.720 INFO:tasks.ceph:role: osd.1 2024-09-18T01:01:43.720 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:01:43.720 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T01:01:43.794 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:01:43.794 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:01:43.795 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:01:43.795 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:01:43.795 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:01:43.795 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:01:43.795 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:01:43.795 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:01:43.795 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:01:43.796 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:01:44.246 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-18T01:01:44.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-18T01:01:44.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T01:01:44.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T01:01:44.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T01:01:44.468 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:44.468 INFO:tasks.ceph:role: osd.2 2024-09-18T01:01:44.468 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:01:44.468 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T01:01:44.542 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:01:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:01:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:01:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:01:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:01:44.543 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:01:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:01:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:01:44.543 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:01:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:01:45.039 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-18T01:01:45.041 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-18T01:01:45.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T01:01:45.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T01:01:45.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:45.320+0000 7f76bfd45640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T01:01:45.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:45.324+0000 7f76bfd45640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T01:01:45.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:45.324+0000 7f76bfd45640 -1 bdev(0x564dd401b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-18T01:01:45.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:45.324+0000 7f76bfd45640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T01:01:48.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T01:01:48.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:48.377+0000 7fc128a10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T01:01:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:48.378+0000 7fc128a10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T01:01:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:48.378+0000 7fc128a10640 -1 bdev(0x557d0c5db800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-18T01:01:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:48.378+0000 7fc128a10640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T01:01:51.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T01:01:51.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:51.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:51.922+0000 7efd2560d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T01:01:51.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:51.923+0000 7efd2560d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T01:01:51.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:51.923+0000 7efd2560d640 -1 bdev(0x55e94cdc1800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-18T01:01:51.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:01:51.923+0000 7efd2560d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T01:01:54.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T01:01:54.866 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T01:01:54.897 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:54.897 INFO:tasks.ceph:role: osd.3 2024-09-18T01:01:54.897 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:01:54.897 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T01:01:54.973 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:01:54.973 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:01:54.974 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:01:55.441 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-18T01:01:55.443 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-18T01:01:55.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-18T01:01:55.566 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T01:01:55.596 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T01:01:55.662 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:55.662 INFO:tasks.ceph:role: osd.4 2024-09-18T01:01:55.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:01:55.662 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:01:55.735 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:01:55.736 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:01:56.181 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-18T01:01:56.183 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-18T01:01:56.183 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-18T01:01:56.254 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T01:01:56.324 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T01:01:56.388 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-18T01:01:56.388 INFO:tasks.ceph:role: osd.5 2024-09-18T01:01:56.389 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:01:56.389 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T01:01:56.460 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T01:01:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T01:01:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T01:01:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T01:01:56.461 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T01:01:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-18T01:01:56.461 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T01:01:56.461 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T01:01:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T01:01:56.461 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T01:01:56.951 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-18T01:01:56.953 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-18T01:01:56.953 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-18T01:01:57.049 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T01:01:57.081 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:01:57.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:01:57.181+0000 7f3e7760d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T01:01:57.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:01:57.185+0000 7f3e7760d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T01:01:57.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:01:57.185+0000 7f3e7760d640 -1 bdev(0x5563304b3800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-18T01:01:57.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:01:57.185+0000 7f3e7760d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T01:02:00.365 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T01:02:00.394 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:02:00.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:00.482+0000 7f6905a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T01:02:00.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:00.483+0000 7f6905a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T01:02:00.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:00.483+0000 7f6905a0f640 -1 bdev(0x556b63e8b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-18T01:02:00.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:00.483+0000 7f6905a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T01:02:03.658 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T01:02:03.688 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:02:03.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:03.777+0000 7febf1da8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T01:02:03.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:03.777+0000 7febf1da8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T01:02:03.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:03.777+0000 7febf1da8640 -1 bdev(0x557461293800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-18T01:02:03.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-18T01:02:03.777+0000 7febf1da8640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T01:02:06.690 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T01:02:06.716 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T01:02:06.716 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:02:06.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T01:02:06.748 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:02:06.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T01:02:06.819 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:02:06.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T01:02:06.891 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:02:06.891 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T01:02:06.917 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:02:06.917 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T01:02:06.983 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:02:06.983 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T01:02:07.048 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:02:07.048 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T01:02:07.113 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-18T01:02:07.113 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T01:02:07.133 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T01:02:07.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T01:02:07.136 DEBUG:teuthology.orchestra.run.smithi098:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T01:02:07.162 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-09-18T01:02:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB5Jupm3swoExAAFiEYF6lDVClP2pSqqGXgDA== 2024-09-18T01:02:07.162 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-09-18T01:02:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB8Jupm76aSFhAAbiNd01N4TvItlXcxb4QVKg== 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB/Jupmay8ONxAAtzsusnfvNb/XMfg2//1lpA== 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB2JupmmrEfLxAAUz9KT72MQ3oCUWS+oP8iuQ== 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCFJupm1YLfChAAibOLuMdaBUGO8+8SWnlI8A== 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCIJupmq0XUHBAALVNrPebjsVtwgYwMTCDMeA== 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCLJupmCQVhLhAAxhLXMYcC0bDswxsQpYkafA== 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-18T01:02:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB2Jupm+HejMhAAyzYGvpX2QMx9WokiO+SXrw== 2024-09-18T01:02:07.176 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.0] 2024-09-18T01:02:07.176 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQB5Jupm3swoExAAFiEYF6lDVClP2pSqqGXgDA== 2024-09-18T01:02:07.176 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.1] 2024-09-18T01:02:07.176 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQB8Jupm76aSFhAAbiNd01N4TvItlXcxb4QVKg== 2024-09-18T01:02:07.176 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.2] 2024-09-18T01:02:07.176 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQB/Jupmay8ONxAAtzsusnfvNb/XMfg2//1lpA== 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.x] 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQB2JupmmrEfLxAAUz9KT72MQ3oCUWS+oP8iuQ== 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.3] 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQCFJupm1YLfChAAibOLuMdaBUGO8+8SWnlI8A== 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.4] 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQCIJupmq0XUHBAALVNrPebjsVtwgYwMTCDMeA== 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.5] 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQCLJupmCQVhLhAAxhLXMYcC0bDswxsQpYkafA== 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-18T01:02:07.177 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQB2Jupm+HejMhAAyzYGvpX2QMx9WokiO+SXrw== 2024-09-18T01:02:07.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.272 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.327 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T01:02:07.381 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T01:02:07.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.434 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.486 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.537 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.539 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T01:02:07.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T01:02:07.592 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T01:02:07.645 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T01:02:07.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T01:02:07.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T01:02:07.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T01:02:07.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T01:02:07.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T01:02:07.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T01:02:07.982 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T01:02:08.008 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T01:02:08.105 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T01:02:08.129 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:02:08.132 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T01:02:08.184 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T01:02:08.185 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T01:02:08.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T01:02:08.188 INFO:tasks.ceph.mon.a:Started 2024-09-18T01:02:08.188 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T01:02:08.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T01:02:08.190 INFO:tasks.ceph.mon.c:Started 2024-09-18T01:02:08.190 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T01:02:08.190 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T01:02:08.226 INFO:tasks.ceph.mon.b:Started 2024-09-18T01:02:08.226 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T01:02:08.227 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T01:02:08.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T01:02:08.229 INFO:tasks.ceph.mgr.x:Started 2024-09-18T01:02:08.229 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T01:02:08.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T01:02:13.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-18T01:02:13.718+0000 7fc1596e5640 0 --2- 172.21.15.31:0/3984789548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc140003a00 0x7fc140003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T01:02:13.800 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-18T01:02:13.813 INFO:tasks.ceph:check_enable_crimson: False 2024-09-18T01:02:13.813 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T01:02:13.813 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:02:13.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T01:02:13.842 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:02:13.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T01:02:13.909 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-18T01:02:13.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T01:02:13.975 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:02:13.975 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T01:02:14.006 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:02:14.006 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T01:02:14.075 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-18T01:02:14.075 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T01:02:14.143 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd new 4563fb16-8745-4e75-b654-1e96403d91b9 0 2024-09-18T01:02:14.372 INFO:teuthology.orchestra.run.smithi098.stdout:0 2024-09-18T01:02:14.384 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd new 634965cb-d444-4721-b34a-afbc49105cab 1 2024-09-18T01:02:14.401 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:14.397+0000 7f0a707ef040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T01:02:14.442 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:14.440+0000 7f0a707ef040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T01:02:14.565 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-18T01:02:14.576 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd new 83443eab-2fa4-4c7e-a459-edb1bb0ed8d0 2 2024-09-18T01:02:14.589 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:14.587+0000 7f0a707ef040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T01:02:14.642 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:14.641+0000 7f0a707ef040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T01:02:14.771 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-18T01:02:14.782 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd new 5797982a-71c1-4d6a-9604-00fbc6bd16d9 3 2024-09-18T01:02:14.812 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:14.811+0000 7f0a707ef040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T01:02:14.928 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:14.927+0000 7f0a707ef040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T01:02:14.967 INFO:teuthology.orchestra.run.smithi098.stdout:3 2024-09-18T01:02:14.978 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd new f79e3329-5481-4f33-8d28-6386af186492 4 2024-09-18T01:02:15.197 INFO:teuthology.orchestra.run.smithi098.stdout:4 2024-09-18T01:02:15.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd new 8c61d2ec-b111-4783-a016-336ed33a0694 5 2024-09-18T01:02:15.272 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:15.271+0000 7f0a707ef040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T01:02:15.327 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:15.325+0000 7f0a707ef040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T01:02:15.386 INFO:teuthology.orchestra.run.smithi098.stdout:5 2024-09-18T01:02:15.397 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T01:02:15.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T01:02:15.400 INFO:tasks.ceph.osd.0:Started 2024-09-18T01:02:15.400 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T01:02:15.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T01:02:15.402 INFO:tasks.ceph.osd.1:Started 2024-09-18T01:02:15.403 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T01:02:15.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T01:02:15.405 INFO:tasks.ceph.osd.2:Started 2024-09-18T01:02:15.405 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T01:02:15.405 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T01:02:15.408 INFO:tasks.ceph.osd.3:Started 2024-09-18T01:02:15.408 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T01:02:15.408 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T01:02:15.411 INFO:tasks.ceph.osd.4:Started 2024-09-18T01:02:15.411 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T01:02:15.411 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T01:02:15.414 INFO:tasks.ceph.osd.5:Started 2024-09-18T01:02:15.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:02:15.488 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:15.486+0000 7f0a707ef040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T01:02:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:15.601 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:02:15.381369+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T01:02:15.381369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:02:15.612 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T01:02:15.612 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T01:02:16.217 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T01:02:16.388 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-18T01:02:16.388 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":8,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:02:15.381369+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T01:02:15.381369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:02:16.397 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-18T01:02:16.457 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:16.456+0000 7f0a707ef040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T01:02:16.509 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:16.508+0000 7f0a707ef040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T01:02:16.595 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:16.594+0000 7fb4cc019640 -1 Falling back to public interface 2024-09-18T01:02:16.615 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:16.614+0000 7fb3ae558640 -1 Falling back to public interface 2024-09-18T01:02:16.854 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:16.852+0000 7f66c3a0c640 -1 Falling back to public interface 2024-09-18T01:02:16.857 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:16.856+0000 7fc6c8e0f640 -1 Falling back to public interface 2024-09-18T01:02:16.873 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:16.872+0000 7f0a707ef040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T01:02:16.957 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:16.955+0000 7f0a707ef040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T01:02:17.061 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.060+0000 7f0a707ef040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T01:02:17.098 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:17.097+0000 7fb186129640 -1 Falling back to public interface 2024-09-18T01:02:17.107 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.106+0000 7f0a707ef040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T01:02:17.221 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.220+0000 7f0a707ef040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T01:02:17.273 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.271+0000 7f0a707ef040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T01:02:17.317 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.316+0000 7f0a707ef040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T01:02:17.356 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:17.355+0000 7f694422c640 -1 Falling back to public interface 2024-09-18T01:02:17.432 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.430+0000 7f0a707ef040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T01:02:17.481 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.480+0000 7f0a707ef040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T01:02:17.531 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.530+0000 7f0a707ef040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T01:02:17.597 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.596+0000 7f0a707ef040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T01:02:17.642 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.640+0000 7f0a707ef040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T01:02:17.686 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:17.685+0000 7f0a707ef040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T01:02:18.335 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:18.333+0000 7f0a707ef040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T01:02:19.347 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:19.346+0000 7f0a110b7640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T01:02:19.348 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:02:19.346+0000 7f0a110b7640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T01:02:20.362 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:20.361+0000 7fb4cc019640 -1 osd.1 0 log_to_monitors true 2024-09-18T01:02:20.411 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:20.409+0000 7fb3ae558640 -1 osd.2 0 log_to_monitors true 2024-09-18T01:02:20.686 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:20.685+0000 7f66c3a0c640 -1 osd.5 0 log_to_monitors true 2024-09-18T01:02:20.855 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:20.853+0000 7fb186129640 -1 osd.0 0 log_to_monitors true 2024-09-18T01:02:20.926 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:20.925+0000 7fc6c8e0f640 -1 osd.3 0 log_to_monitors true 2024-09-18T01:02:21.123 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:21.121+0000 7f694422c640 -1 osd.4 0 log_to_monitors true 2024-09-18T01:02:23.002 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T01:02:23.288 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:23.287+0000 7f66bf3d9640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T01:02:23.302 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:23.301+0000 7f66b71f9640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:02:23.350 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:23.348+0000 7f69409cf640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T01:02:23.366 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:23.365+0000 7f6937fdb640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:02:23.379 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:23.378+0000 7fc6c46ed640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T01:02:23.389 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:23.388+0000 7fc6bc50d640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:02:23.425 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-18T01:02:23.425 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:02:22.346556+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T01:02:15.381369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:02:23.441 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-18T01:02:23.449 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:23.447+0000 7fb4c7926640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T01:02:23.452 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:23.450+0000 7fb1820b8640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T01:02:23.460 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:23.458+0000 7fb4bf746640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:02:23.465 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:23.463+0000 7fb3aa4e7640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T01:02:23.472 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:23.469+0000 7fb179ed8640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:02:23.475 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:23.473+0000 7fb3a2307640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T01:02:30.047 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T01:02:30.343 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-18T01:02:30.343 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":14,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:02:27.295244+0000","last_up_change":"2024-09-18T01:02:24.290620+0000","last_in_change":"2024-09-18T01:02:15.381369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:02:24.388064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6817","nonce":1626166637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6819","nonce":1626166637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6823","nonce":1626166637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6821","nonce":1626166637}]},"public_addr":"172.21.15.31:6817/1626166637","cluster_addr":"172.21.15.31:6819/1626166637","heartbeat_back_addr":"172.21.15.31:6823/1626166637","heartbeat_front_addr":"172.21.15.31:6821/1626166637","state":["exists","up"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6801","nonce":2210970259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6803","nonce":2210970259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6807","nonce":2210970259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6805","nonce":2210970259}]},"public_addr":"172.21.15.31:6801/2210970259","cluster_addr":"172.21.15.31:6803/2210970259","heartbeat_back_addr":"172.21.15.31:6807/2210970259","heartbeat_front_addr":"172.21.15.31:6805/2210970259","state":["exists","up"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6809","nonce":617133675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6811","nonce":617133675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6815","nonce":617133675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6813","nonce":617133675}]},"public_addr":"172.21.15.31:6809/617133675","cluster_addr":"172.21.15.31:6811/617133675","heartbeat_back_addr":"172.21.15.31:6815/617133675","heartbeat_front_addr":"172.21.15.31:6813/617133675","state":["exists","up"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6809","nonce":151031464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6811","nonce":151031464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6815","nonce":151031464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6813","nonce":151031464}]},"public_addr":"172.21.15.98:6809/151031464","cluster_addr":"172.21.15.98:6811/151031464","heartbeat_back_addr":"172.21.15.98:6815/151031464","heartbeat_front_addr":"172.21.15.98:6813/151031464","state":["exists","up"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6817","nonce":1499036965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6819","nonce":1499036965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6823","nonce":1499036965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6821","nonce":1499036965}]},"public_addr":"172.21.15.98:6817/1499036965","cluster_addr":"172.21.15.98:6819/1499036965","heartbeat_back_addr":"172.21.15.98:6823/1499036965","heartbeat_front_addr":"172.21.15.98:6821/1499036965","state":["exists","up"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6801","nonce":2310498524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6803","nonce":2310498524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6807","nonce":2310498524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6805","nonce":2310498524}]},"public_addr":"172.21.15.98:6801/2310498524","cluster_addr":"172.21.15.98:6803/2310498524","heartbeat_back_addr":"172.21.15.98:6807/2310498524","heartbeat_front_addr":"172.21.15.98:6805/2310498524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.877700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.360930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.403205+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.952123+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:22.099545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.694132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:02:30.355 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-18T01:02:30.355 INFO:tasks.ceph:Creating RBD pool 2024-09-18T01:02:30.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T01:02:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2024-09-18T01:02:31.385 DEBUG:teuthology.orchestra.run.smithi031:> rbd --cluster ceph pool init rbd 2024-09-18T01:02:34.390 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T01:02:34.391 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T01:02:34.391 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T01:02:34.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T01:02:34.392 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T01:02:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:34.752 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3256274330},{"type":"v1","addr":"172.21.15.98:6825","nonce":3256274330}]},"active_addr":"172.21.15.98:6825/3256274330","active_change":"2024-09-18T01:02:18.336405+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":2072951880}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":16829169}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":1797578697}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":3614903616}]}]} 2024-09-18T01:02:34.755 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T01:02:34.755 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T01:02:34.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:02:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:35.043 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:02:34.372294+0000","last_up_change":"2024-09-18T01:02:24.290620+0000","last_in_change":"2024-09-18T01:02:15.381369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:02:24.388064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:02:30.676199+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6817","nonce":1626166637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6819","nonce":1626166637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6823","nonce":1626166637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6821","nonce":1626166637}]},"public_addr":"172.21.15.31:6817/1626166637","cluster_addr":"172.21.15.31:6819/1626166637","heartbeat_back_addr":"172.21.15.31:6823/1626166637","heartbeat_front_addr":"172.21.15.31:6821/1626166637","state":["exists","up"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6801","nonce":2210970259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6803","nonce":2210970259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6807","nonce":2210970259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6805","nonce":2210970259}]},"public_addr":"172.21.15.31:6801/2210970259","cluster_addr":"172.21.15.31:6803/2210970259","heartbeat_back_addr":"172.21.15.31:6807/2210970259","heartbeat_front_addr":"172.21.15.31:6805/2210970259","state":["exists","up"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6809","nonce":617133675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6811","nonce":617133675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6815","nonce":617133675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6813","nonce":617133675}]},"public_addr":"172.21.15.31:6809/617133675","cluster_addr":"172.21.15.31:6811/617133675","heartbeat_back_addr":"172.21.15.31:6815/617133675","heartbeat_front_addr":"172.21.15.31:6813/617133675","state":["exists","up"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6809","nonce":151031464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6811","nonce":151031464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6815","nonce":151031464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6813","nonce":151031464}]},"public_addr":"172.21.15.98:6809/151031464","cluster_addr":"172.21.15.98:6811/151031464","heartbeat_back_addr":"172.21.15.98:6815/151031464","heartbeat_front_addr":"172.21.15.98:6813/151031464","state":["exists","up"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6817","nonce":1499036965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6819","nonce":1499036965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6823","nonce":1499036965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6821","nonce":1499036965}]},"public_addr":"172.21.15.98:6817/1499036965","cluster_addr":"172.21.15.98:6819/1499036965","heartbeat_back_addr":"172.21.15.98:6823/1499036965","heartbeat_front_addr":"172.21.15.98:6821/1499036965","state":["exists","up"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6801","nonce":2310498524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6803","nonce":2310498524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6807","nonce":2310498524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6805","nonce":2310498524}]},"public_addr":"172.21.15.98:6801/2310498524","cluster_addr":"172.21.15.98:6803/2310498524","heartbeat_back_addr":"172.21.15.98:6807/2310498524","heartbeat_front_addr":"172.21.15.98:6805/2310498524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.877700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.360930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.403205+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.952123+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:22.099545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.694132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:02:35.055 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T01:02:35.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:02:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:35.344 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:02:34.372294+0000","last_up_change":"2024-09-18T01:02:24.290620+0000","last_in_change":"2024-09-18T01:02:15.381369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:02:24.388064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:02:30.676199+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6817","nonce":1626166637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6819","nonce":1626166637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6823","nonce":1626166637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6821","nonce":1626166637}]},"public_addr":"172.21.15.31:6817/1626166637","cluster_addr":"172.21.15.31:6819/1626166637","heartbeat_back_addr":"172.21.15.31:6823/1626166637","heartbeat_front_addr":"172.21.15.31:6821/1626166637","state":["exists","up"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6801","nonce":2210970259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6803","nonce":2210970259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6807","nonce":2210970259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6805","nonce":2210970259}]},"public_addr":"172.21.15.31:6801/2210970259","cluster_addr":"172.21.15.31:6803/2210970259","heartbeat_back_addr":"172.21.15.31:6807/2210970259","heartbeat_front_addr":"172.21.15.31:6805/2210970259","state":["exists","up"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6809","nonce":617133675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6811","nonce":617133675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6815","nonce":617133675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6813","nonce":617133675}]},"public_addr":"172.21.15.31:6809/617133675","cluster_addr":"172.21.15.31:6811/617133675","heartbeat_back_addr":"172.21.15.31:6815/617133675","heartbeat_front_addr":"172.21.15.31:6813/617133675","state":["exists","up"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6809","nonce":151031464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6811","nonce":151031464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6815","nonce":151031464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6813","nonce":151031464}]},"public_addr":"172.21.15.98:6809/151031464","cluster_addr":"172.21.15.98:6811/151031464","heartbeat_back_addr":"172.21.15.98:6815/151031464","heartbeat_front_addr":"172.21.15.98:6813/151031464","state":["exists","up"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6817","nonce":1499036965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6819","nonce":1499036965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6823","nonce":1499036965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6821","nonce":1499036965}]},"public_addr":"172.21.15.98:6817/1499036965","cluster_addr":"172.21.15.98:6819/1499036965","heartbeat_back_addr":"172.21.15.98:6823/1499036965","heartbeat_front_addr":"172.21.15.98:6821/1499036965","state":["exists","up"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6801","nonce":2310498524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6803","nonce":2310498524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6807","nonce":2310498524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6805","nonce":2310498524}]},"public_addr":"172.21.15.98:6801/2310498524","cluster_addr":"172.21.15.98:6803/2310498524","heartbeat_back_addr":"172.21.15.98:6807/2310498524","heartbeat_front_addr":"172.21.15.98:6805/2310498524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.877700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.360930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.403205+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.952123+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:22.099545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.694132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:02:35.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T01:02:35.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T01:02:35.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T01:02:35.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T01:02:35.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T01:02:35.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T01:02:35.520 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-18T01:02:35.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:02:35.521 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-18T01:02:35.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:02:35.565 INFO:teuthology.orchestra.run.smithi031.stdout:47244640262 2024-09-18T01:02:35.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:02:35.568 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-18T01:02:35.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:02:35.573 INFO:teuthology.orchestra.run.smithi031.stdout:47244640262 2024-09-18T01:02:35.574 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:02:35.577 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-18T01:02:35.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:02:35.905 INFO:teuthology.orchestra.run.smithi031.stdout:47244640259 2024-09-18T01:02:35.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.3 2024-09-18T01:02:35.921 INFO:teuthology.orchestra.run.smithi031.stdout:47244640258 2024-09-18T01:02:35.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.5 2024-09-18T01:02:36.033 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-18T01:02:36.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.4 2024-09-18T01:02:36.059 INFO:teuthology.orchestra.run.smithi031.stdout:47244640258 2024-09-18T01:02:36.062 INFO:teuthology.orchestra.run.smithi031.stdout:47244640259 2024-09-18T01:02:36.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.2 2024-09-18T01:02:36.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.0 2024-09-18T01:02:36.086 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-18T01:02:36.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.1 2024-09-18T01:02:36.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:02:36.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:02:37.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:02:37.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:02:37.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:02:37.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:02:37.329 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-18T01:02:37.330 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-18T01:02:37.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.5 2024-09-18T01:02:37.345 DEBUG:teuthology.parallel:result is None 2024-09-18T01:02:37.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2024-09-18T01:02:37.352 DEBUG:teuthology.parallel:result is None 2024-09-18T01:02:37.456 INFO:teuthology.orchestra.run.smithi031.stdout:47244640262 2024-09-18T01:02:37.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.4 2024-09-18T01:02:37.470 DEBUG:teuthology.parallel:result is None 2024-09-18T01:02:37.481 INFO:teuthology.orchestra.run.smithi031.stdout:47244640262 2024-09-18T01:02:37.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.1 2024-09-18T01:02:37.494 DEBUG:teuthology.parallel:result is None 2024-09-18T01:02:37.561 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-18T01:02:37.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.2 2024-09-18T01:02:37.574 DEBUG:teuthology.parallel:result is None 2024-09-18T01:02:37.576 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-18T01:02:37.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2024-09-18T01:02:37.593 DEBUG:teuthology.parallel:result is None 2024-09-18T01:02:37.593 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T01:02:37.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:02:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:02:37.931 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-18T01:02:36.344801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981121"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427654+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:02:34.427654+0000","last_peered":"2024-09-18T01:02:34.427654+0000","last_clean":"2024-09-18T01:02:34.427654+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:02:34.427654+0000","last_undegraded":"2024-09-18T01:02:34.427654+0000","last_fullsized":"2024-09-18T01:02:34.427654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383841+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:02:34.383841+0000","last_peered":"2024-09-18T01:02:34.383841+0000","last_clean":"2024-09-18T01:02:34.383841+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:02:34.383841+0000","last_undegraded":"2024-09-18T01:02:34.383841+0000","last_fullsized":"2024-09-18T01:02:34.383841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427875+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:02:34.427875+0000","last_peered":"2024-09-18T01:02:34.427875+0000","last_clean":"2024-09-18T01:02:34.427875+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:02:34.427875+0000","last_undegraded":"2024-09-18T01:02:34.427875+0000","last_fullsized":"2024-09-18T01:02:34.427875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383771+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:02:34.383771+0000","last_peered":"2024-09-18T01:02:34.383771+0000","last_clean":"2024-09-18T01:02:34.383771+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:02:34.383771+0000","last_undegraded":"2024-09-18T01:02:34.383771+0000","last_fullsized":"2024-09-18T01:02:34.383771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.382376+0000","last_change":"2024-09-18T01:02:34.382376+0000","last_active":"2024-09-18T01:02:34.382376+0000","last_peered":"2024-09-18T01:02:34.382376+0000","last_clean":"2024-09-18T01:02:34.382376+0000","last_became_active":"2024-09-18T01:02:32.377565+0000","last_became_peered":"2024-09-18T01:02:32.377565+0000","last_unstale":"2024-09-18T01:02:34.382376+0000","last_undegraded":"2024-09-18T01:02:34.382376+0000","last_fullsized":"2024-09-18T01:02:34.382376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:56.817912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001120694,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.378495+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:02:34.378495+0000","last_peered":"2024-09-18T01:02:34.378495+0000","last_clean":"2024-09-18T01:02:34.378495+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:02:34.378495+0000","last_undegraded":"2024-09-18T01:02:34.378495+0000","last_fullsized":"2024-09-18T01:02:34.378495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427587+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:02:34.427587+0000","last_peered":"2024-09-18T01:02:34.427587+0000","last_clean":"2024-09-18T01:02:34.427587+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:02:34.427587+0000","last_undegraded":"2024-09-18T01:02:34.427587+0000","last_fullsized":"2024-09-18T01:02:34.427587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.381487+0000","last_change":"2024-09-18T01:02:34.381765+0000","last_active":"2024-09-18T01:02:34.381487+0000","last_peered":"2024-09-18T01:02:34.381487+0000","last_clean":"2024-09-18T01:02:34.381487+0000","last_became_active":"2024-09-18T01:02:32.377256+0000","last_became_peered":"2024-09-18T01:02:32.377256+0000","last_unstale":"2024-09-18T01:02:34.381487+0000","last_undegraded":"2024-09-18T01:02:34.381487+0000","last_fullsized":"2024-09-18T01:02:34.381487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:00.553619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050918999999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":171,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427371+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:02:34.427371+0000","last_peered":"2024-09-18T01:02:34.427371+0000","last_clean":"2024-09-18T01:02:34.427371+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:02:34.427371+0000","last_undegraded":"2024-09-18T01:02:34.427371+0000","last_fullsized":"2024-09-18T01:02:34.427371+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:02:37.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:02:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:02:38.232 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-18T01:02:36.344801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981121"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427654+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:02:34.427654+0000","last_peered":"2024-09-18T01:02:34.427654+0000","last_clean":"2024-09-18T01:02:34.427654+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:02:34.427654+0000","last_undegraded":"2024-09-18T01:02:34.427654+0000","last_fullsized":"2024-09-18T01:02:34.427654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383841+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:02:34.383841+0000","last_peered":"2024-09-18T01:02:34.383841+0000","last_clean":"2024-09-18T01:02:34.383841+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:02:34.383841+0000","last_undegraded":"2024-09-18T01:02:34.383841+0000","last_fullsized":"2024-09-18T01:02:34.383841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427875+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:02:34.427875+0000","last_peered":"2024-09-18T01:02:34.427875+0000","last_clean":"2024-09-18T01:02:34.427875+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:02:34.427875+0000","last_undegraded":"2024-09-18T01:02:34.427875+0000","last_fullsized":"2024-09-18T01:02:34.427875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383771+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:02:34.383771+0000","last_peered":"2024-09-18T01:02:34.383771+0000","last_clean":"2024-09-18T01:02:34.383771+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:02:34.383771+0000","last_undegraded":"2024-09-18T01:02:34.383771+0000","last_fullsized":"2024-09-18T01:02:34.383771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.382376+0000","last_change":"2024-09-18T01:02:34.382376+0000","last_active":"2024-09-18T01:02:34.382376+0000","last_peered":"2024-09-18T01:02:34.382376+0000","last_clean":"2024-09-18T01:02:34.382376+0000","last_became_active":"2024-09-18T01:02:32.377565+0000","last_became_peered":"2024-09-18T01:02:32.377565+0000","last_unstale":"2024-09-18T01:02:34.382376+0000","last_undegraded":"2024-09-18T01:02:34.382376+0000","last_fullsized":"2024-09-18T01:02:34.382376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:56.817912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001120694,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.378495+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:02:34.378495+0000","last_peered":"2024-09-18T01:02:34.378495+0000","last_clean":"2024-09-18T01:02:34.378495+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:02:34.378495+0000","last_undegraded":"2024-09-18T01:02:34.378495+0000","last_fullsized":"2024-09-18T01:02:34.378495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427587+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:02:34.427587+0000","last_peered":"2024-09-18T01:02:34.427587+0000","last_clean":"2024-09-18T01:02:34.427587+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:02:34.427587+0000","last_undegraded":"2024-09-18T01:02:34.427587+0000","last_fullsized":"2024-09-18T01:02:34.427587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.381487+0000","last_change":"2024-09-18T01:02:34.381765+0000","last_active":"2024-09-18T01:02:34.381487+0000","last_peered":"2024-09-18T01:02:34.381487+0000","last_clean":"2024-09-18T01:02:34.381487+0000","last_became_active":"2024-09-18T01:02:32.377256+0000","last_became_peered":"2024-09-18T01:02:32.377256+0000","last_unstale":"2024-09-18T01:02:34.381487+0000","last_undegraded":"2024-09-18T01:02:34.381487+0000","last_fullsized":"2024-09-18T01:02:34.381487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:00.553619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050918999999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":171,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427371+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:02:34.427371+0000","last_peered":"2024-09-18T01:02:34.427371+0000","last_clean":"2024-09-18T01:02:34.427371+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:02:34.427371+0000","last_undegraded":"2024-09-18T01:02:34.427371+0000","last_fullsized":"2024-09-18T01:02:34.427371+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:02:38.233 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T01:02:38.233 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T01:02:38.234 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T01:02:38.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T01:02:38.563 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:38.564 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T01:02:38.577 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T01:02:38.577 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-18T01:02:38.588 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-18T01:02:38.589 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-18T01:02:38.589 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-18T01:02:38.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-18T01:02:38.896 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-18T01:02:38.896 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T01:02:38.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:02:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:02:39.201 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-18T01:02:38.345125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.977484"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427654+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:02:34.427654+0000","last_peered":"2024-09-18T01:02:34.427654+0000","last_clean":"2024-09-18T01:02:34.427654+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:02:34.427654+0000","last_undegraded":"2024-09-18T01:02:34.427654+0000","last_fullsized":"2024-09-18T01:02:34.427654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383841+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:02:34.383841+0000","last_peered":"2024-09-18T01:02:34.383841+0000","last_clean":"2024-09-18T01:02:34.383841+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:02:34.383841+0000","last_undegraded":"2024-09-18T01:02:34.383841+0000","last_fullsized":"2024-09-18T01:02:34.383841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427875+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:02:34.427875+0000","last_peered":"2024-09-18T01:02:34.427875+0000","last_clean":"2024-09-18T01:02:34.427875+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:02:34.427875+0000","last_undegraded":"2024-09-18T01:02:34.427875+0000","last_fullsized":"2024-09-18T01:02:34.427875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383771+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:02:34.383771+0000","last_peered":"2024-09-18T01:02:34.383771+0000","last_clean":"2024-09-18T01:02:34.383771+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:02:34.383771+0000","last_undegraded":"2024-09-18T01:02:34.383771+0000","last_fullsized":"2024-09-18T01:02:34.383771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.382376+0000","last_change":"2024-09-18T01:02:34.382376+0000","last_active":"2024-09-18T01:02:34.382376+0000","last_peered":"2024-09-18T01:02:34.382376+0000","last_clean":"2024-09-18T01:02:34.382376+0000","last_became_active":"2024-09-18T01:02:32.377565+0000","last_became_peered":"2024-09-18T01:02:32.377565+0000","last_unstale":"2024-09-18T01:02:34.382376+0000","last_undegraded":"2024-09-18T01:02:34.382376+0000","last_fullsized":"2024-09-18T01:02:34.382376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:56.817912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001120694,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.378495+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:02:34.378495+0000","last_peered":"2024-09-18T01:02:34.378495+0000","last_clean":"2024-09-18T01:02:34.378495+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:02:34.378495+0000","last_undegraded":"2024-09-18T01:02:34.378495+0000","last_fullsized":"2024-09-18T01:02:34.378495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427587+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:02:34.427587+0000","last_peered":"2024-09-18T01:02:34.427587+0000","last_clean":"2024-09-18T01:02:34.427587+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:02:34.427587+0000","last_undegraded":"2024-09-18T01:02:34.427587+0000","last_fullsized":"2024-09-18T01:02:34.427587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.381487+0000","last_change":"2024-09-18T01:02:34.381765+0000","last_active":"2024-09-18T01:02:34.381487+0000","last_peered":"2024-09-18T01:02:34.381487+0000","last_clean":"2024-09-18T01:02:34.381487+0000","last_became_active":"2024-09-18T01:02:32.377256+0000","last_became_peered":"2024-09-18T01:02:32.377256+0000","last_unstale":"2024-09-18T01:02:34.381487+0000","last_undegraded":"2024-09-18T01:02:34.381487+0000","last_fullsized":"2024-09-18T01:02:34.381487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:00.553619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050918999999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":171,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427371+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:02:34.427371+0000","last_peered":"2024-09-18T01:02:34.427371+0000","last_clean":"2024-09-18T01:02:34.427371+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:02:34.427371+0000","last_undegraded":"2024-09-18T01:02:34.427371+0000","last_fullsized":"2024-09-18T01:02:34.427371+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:02:39.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:02:39.488 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:02:39.500 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-18T01:02:38.345125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163324,"kb_used_data":1128,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067716,"statfs":{"total":579820584960,"available":579653341184,"internally_reserved":0,"allocated":1155072,"data_stored":1521508,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.977484"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427654+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:02:34.427654+0000","last_peered":"2024-09-18T01:02:34.427654+0000","last_clean":"2024-09-18T01:02:34.427654+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:02:34.427654+0000","last_undegraded":"2024-09-18T01:02:34.427654+0000","last_fullsized":"2024-09-18T01:02:34.427654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383841+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:02:34.383841+0000","last_peered":"2024-09-18T01:02:34.383841+0000","last_clean":"2024-09-18T01:02:34.383841+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:02:34.383841+0000","last_undegraded":"2024-09-18T01:02:34.383841+0000","last_fullsized":"2024-09-18T01:02:34.383841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427875+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:02:34.427875+0000","last_peered":"2024-09-18T01:02:34.427875+0000","last_clean":"2024-09-18T01:02:34.427875+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:02:34.427875+0000","last_undegraded":"2024-09-18T01:02:34.427875+0000","last_fullsized":"2024-09-18T01:02:34.427875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.383771+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:02:34.383771+0000","last_peered":"2024-09-18T01:02:34.383771+0000","last_clean":"2024-09-18T01:02:34.383771+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:02:34.383771+0000","last_undegraded":"2024-09-18T01:02:34.383771+0000","last_fullsized":"2024-09-18T01:02:34.383771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.382376+0000","last_change":"2024-09-18T01:02:34.382376+0000","last_active":"2024-09-18T01:02:34.382376+0000","last_peered":"2024-09-18T01:02:34.382376+0000","last_clean":"2024-09-18T01:02:34.382376+0000","last_became_active":"2024-09-18T01:02:32.377565+0000","last_became_peered":"2024-09-18T01:02:32.377565+0000","last_unstale":"2024-09-18T01:02:34.382376+0000","last_undegraded":"2024-09-18T01:02:34.382376+0000","last_fullsized":"2024-09-18T01:02:34.382376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:56.817912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001120694,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.378495+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:02:34.378495+0000","last_peered":"2024-09-18T01:02:34.378495+0000","last_clean":"2024-09-18T01:02:34.378495+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:02:34.378495+0000","last_undegraded":"2024-09-18T01:02:34.378495+0000","last_fullsized":"2024-09-18T01:02:34.378495+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427587+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:02:34.427587+0000","last_peered":"2024-09-18T01:02:34.427587+0000","last_clean":"2024-09-18T01:02:34.427587+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:02:34.427587+0000","last_undegraded":"2024-09-18T01:02:34.427587+0000","last_fullsized":"2024-09-18T01:02:34.427587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.381487+0000","last_change":"2024-09-18T01:02:34.381765+0000","last_active":"2024-09-18T01:02:34.381487+0000","last_peered":"2024-09-18T01:02:34.381487+0000","last_clean":"2024-09-18T01:02:34.381487+0000","last_became_active":"2024-09-18T01:02:32.377256+0000","last_became_peered":"2024-09-18T01:02:32.377256+0000","last_unstale":"2024-09-18T01:02:34.381487+0000","last_undegraded":"2024-09-18T01:02:34.381487+0000","last_fullsized":"2024-09-18T01:02:34.381487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:00.553619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050918999999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":171,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T01:02:34.427371+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:02:34.427371+0000","last_peered":"2024-09-18T01:02:34.427371+0000","last_clean":"2024-09-18T01:02:34.427371+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:02:34.427371+0000","last_undegraded":"2024-09-18T01:02:34.427371+0000","last_fullsized":"2024-09-18T01:02:34.427371+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640262,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":56789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":176128,"data_stored":56808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":647157,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:02:39.501 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T01:02:39.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 19 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 62b46b83-8ef2-466f-b28b-2d97f265422e 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-18T01:02:13.493841+0000 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-18T01:02:38.273255+0000 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 4 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-18T01:02:39.788 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client jewel 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 14 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 6 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/1626166637,v1:172.21.15.31:6817/1626166637] [v2:172.21.15.31:6818/1626166637,v1:172.21.15.31:6819/1626166637] exists,up 4563fb16-8745-4e75-b654-1e96403d91b9 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/2210970259,v1:172.21.15.31:6801/2210970259] [v2:172.21.15.31:6802/2210970259,v1:172.21.15.31:6803/2210970259] exists,up 634965cb-d444-4721-b34a-afbc49105cab 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/617133675,v1:172.21.15.31:6809/617133675] [v2:172.21.15.31:6810/617133675,v1:172.21.15.31:6811/617133675] exists,up 83443eab-2fa4-4c7e-a459-edb1bb0ed8d0 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/151031464,v1:172.21.15.98:6809/151031464] [v2:172.21.15.98:6810/151031464,v1:172.21.15.98:6811/151031464] exists,up 5797982a-71c1-4d6a-9604-00fbc6bd16d9 2024-09-18T01:02:39.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/1499036965,v1:172.21.15.98:6817/1499036965] [v2:172.21.15.98:6818/1499036965,v1:172.21.15.98:6819/1499036965] exists,up f79e3329-5481-4f33-8d28-6386af186492 2024-09-18T01:02:39.790 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/2310498524,v1:172.21.15.98:6801/2310498524] [v2:172.21.15.98:6802/2310498524,v1:172.21.15.98:6803/2310498524] exists,up 8c61d2ec-b111-4783-a016-336ed33a0694 2024-09-18T01:02:39.800 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/1626166637,v1:172.21.15.31:6817/1626166637] [v2:172.21.15.31:6818/1626166637,v1:172.21.15.31:6819/1626166637] exists,up 4563fb16-8745-4e75-b654-1e96403d91b9', 'osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/2210970259,v1:172.21.15.31:6801/2210970259] [v2:172.21.15.31:6802/2210970259,v1:172.21.15.31:6803/2210970259] exists,up 634965cb-d444-4721-b34a-afbc49105cab', 'osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/617133675,v1:172.21.15.31:6809/617133675] [v2:172.21.15.31:6810/617133675,v1:172.21.15.31:6811/617133675] exists,up 83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/151031464,v1:172.21.15.98:6809/151031464] [v2:172.21.15.98:6810/151031464,v1:172.21.15.98:6811/151031464] exists,up 5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'osd.4 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/1499036965,v1:172.21.15.98:6817/1499036965] [v2:172.21.15.98:6818/1499036965,v1:172.21.15.98:6819/1499036965] exists,up f79e3329-5481-4f33-8d28-6386af186492', 'osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/2310498524,v1:172.21.15.98:6801/2310498524] [v2:172.21.15.98:6802/2310498524,v1:172.21.15.98:6803/2310498524] exists,up 8c61d2ec-b111-4783-a016-336ed33a0694'] 2024-09-18T01:02:39.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-18T01:02:39.901 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mon.a", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster": "ceph", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket_mode": "", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_client_required": "cephx, none", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_cluster_required": "cephx", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_debug": "true", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-18T01:02:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_required": "cephx", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_supported": "cephx", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio": "true", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_poll_ms": "250", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_reap_max": "16", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_async_discard": "false", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_block_size": "4096", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio": "false", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-18T01:02:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_inflight_ios": "false", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_enable_discard": "false", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry": "3", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash": "0", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring": "false", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_hipri": "false", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-18T01:02:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_type": "", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_alloc_size": "1048576", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_allocator": "hybrid", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_buffered_io": "true", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_for_zeros": "false", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-18T01:02:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_compact_log_sync": "false", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_log_runway": "4194304", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_prefetch": "1048576", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_flush_size": "524288", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_log_runway": "1048576", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery": "false", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-18T01:02:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_sync_write": "false", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocation_from_file": "true", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocator": "hybrid", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_multi": "true", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-18T01:02:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_create": "true", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_create": "false", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_path": "", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_size": "0", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_path": "", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_preallocate_file": "false", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_size": "107374182400", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_create": "false", 2024-09-18T01:02:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_path": "", 2024-09-18T01:02:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_size": "100663296", 2024-09-18T01:02:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs": "true", 2024-09-18T01:02:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-18T01:02:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-18T01:02:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune": "true", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-18T01:02:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size": "0", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_type": "2q", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_clone_cow": "true", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-18T01:02:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_mode": "none", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_csum_type": "crc32c", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_freelist": "false", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-18T01:02:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-18T01:02:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-18T01:02:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-18T01:02:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-18T01:02:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-18T01:02:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-18T01:02:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-18T01:02:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_small_allocations": "0", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_read": "true", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_write": "false", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-18T01:02:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fail_eio": "false", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-18T01:02:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount": "false", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount": "false", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-18T01:02:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_ignore_data_csum": "false", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_op_age": "5.000000", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_alloc_size": "0", 2024-09-18T01:02:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size": "0", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_deferred_txc": "32", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size": "0", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_nid_prealloc": "1024", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-18T01:02:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_retry_disk_reads": "3", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cf": "true", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-18T01:02:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_mem": "512", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_tracing": "false", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-18T01:02:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_zero_block_detection": "false", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-18T01:02:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_signatures": "false", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_version": "2", 2024-09-18T01:02:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_signatures": "false", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_version": "2", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_signatures": "false", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_version": "2", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_sign_messages": "true", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "chdir": "", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_acl_type": "", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_asio_thread_count": "2", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_mid": "0.750000", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_size": "16384", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_caps_release_delay": "5", 2024-09-18T01:02:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_check_pool_perm": "true", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_force_sync_read": "false", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_getattr_caps": "false", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_features": "", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_tick_delay": "0", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_remount": "false", 2024-09-18T01:02:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_dirsize_rbytes": "true", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_force_lazyio": "false", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_fs": "", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_release_failure": "false", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_inline_size": "4096", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mds_namespace": "", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_metadata": "", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_gid": "-1", 2024-09-18T01:02:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_timeout": "300", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_uid": "-1", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mountpoint": "/", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_notify_timeout": "10", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc": "true", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty": "104857600", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_objects": "1000", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_size": "209715200", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_target_dirty": "8388608", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_permissions": "true", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota": "true", 2024-09-18T01:02:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota_df": "true", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_bytes": "0", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_periods": "4", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_min": "131072", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_reconnect_stale": "false", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_shutdown_timeout": "30", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_snapdir": ".snap", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_tick_interval": "1", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_trace": "", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_try_dentry_invalidate": "false", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_faked_inos": "false", 2024-09-18T01:02:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_random_mds": "false", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog": "false", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_port": "12201", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_monitors": "default=true", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog": "false", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_level": "info", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "-", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network": "", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network_interface": "", 2024-09-18T01:02:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_isal": "false", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_level": "5", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_winsize": "-15", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zstd_level": "1", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_op_num_threads": "6", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_stat_interval": "0", 2024-09-18T01:02:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_cpu_cores": "", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_num_threads": "0", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location": "", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook": "", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook_timeout": "10", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "daemonize": "false", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "daos_pool": "tank", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_alienstore": "0/5", 2024-09-18T01:02:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_allow_any_pool_priority": "false", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok": "1/5", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok_assert_abort": "false", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asserts_on_shutdown": "true", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_auth": "20/20", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bdev": "1/3", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluefs": "1/5", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluestore": "1/5", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_buffer": "0/1", 2024-09-18T01:02:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ceph_exporter": "1/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephfs_mirror": "0/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephsqlite": "0/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_client": "0/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_compressor": "1/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_context": "0/1", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crimson_interrupt": "0/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crush": "1/1", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crypto": "1/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cyanstore": "0/5", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_deliberately_leak_memory": "false", 2024-09-18T01:02:39.932 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_disable_randomized_ping": "false", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_dpdk": "1/5", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_eventtrace": "1/5", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filer": "0/1", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filestore": "1/3", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_finisher": "1/1", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_fuse": "1/5", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeat_testing_span": "0", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeatmap": "1/5", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_javaclient": "1/5", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journal": "1/3", 2024-09-18T01:02:39.933 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journaler": "0/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_kstore": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_lockdep": "0/1", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mclock": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_balancer": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_locker": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log_expire": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_migrator": "1/5", 2024-09-18T01:02:39.934 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_quiesce": "3/5", 2024-09-18T01:02:39.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_memstore": "1/5", 2024-09-18T01:02:39.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgr": "2/5", 2024-09-18T01:02:39.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgrc": "1/5", 2024-09-18T01:02:39.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mon": "20/20", 2024-09-18T01:02:39.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_monc": "0/10", 2024-09-18T01:02:39.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ms": "1/1", 2024-09-18T01:02:39.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_none": "0/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objclass": "0/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objectcacher": "0/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objecter": "0/1", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_optracker": "0/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_osd": "1/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_paxos": "20/20", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_perfcounter": "1/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_prioritycache": "1/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rados": "0/5", 2024-09-18T01:02:39.936 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd": "0/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_mirror": "0/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_pwl": "0/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_replay": "0/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_refs": "0/0", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_reserver": "1/1", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw": "1/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_access": "1/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_datacache": "1/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_dbstore": "1/5", 2024-09-18T01:02:39.937 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_flight": "1/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_sync": "1/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rocksdb": "4/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_backref": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cache": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cleaner": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_device": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_epm": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_journal": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_lba": "0/5", 2024-09-18T01:02:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_odata": "0/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_omap": "0/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_onode": "0/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_t": "0/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_tm": "0/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_striper": "0/1", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_test": "0/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_throttle": "1/1", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_timer": "0/1", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_tp": "0/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_trace": "1/5", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "device_failure_prediction_mode": "none", 2024-09-18T01:02:39.939 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_graylog": "false", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_journald": "false", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_stderr": "true", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_syslog": "false", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event_tracing": "false", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_addr": "0.0.0.0", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_http_port": "9926", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_prio_limit": "5", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sort_metrics": "true", 2024-09-18T01:02:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_stats_period": "5", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "fake_statfs_for_testing": "0", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "fatal_signal_handlers": "true", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_purge_ops": "10", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_truncate_ops": "128", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_apply_finisher_threads": "1", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_blackhole": "false", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_clone_range": "true", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_snap": "true", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_caller_concurrency": "10", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_collect_device_partition_information": "true", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_commit_timeout": "600.000000", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_inject_read_err": "false", 2024-09-18T01:02:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_omap_check": "false", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_verify_split": "false", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_dump_file": "", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fadvise": "true", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fail_eio": "true", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_shards": "16", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_size": "128", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap": "false", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap_threshold": "4096", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-18T01:02:39.942 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_inject_stall": "0", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_parallel": "false", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_trailing": "false", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_writeahead": "false", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_kill_at": "0", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs": "0", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-18T01:02:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size": "0", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_merge_threshold": "-10", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_odsync_write": "false", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend": "rocksdb", 2024-09-18T01:02:39.944 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend_path": "", 2024-09-18T01:02:39.945 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-18T01:02:39.945 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-18T01:02:39.945 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-18T01:02:39.945 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_timeout": "60", 2024-09-18T01:02:39.945 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_threads": "2", 2024-09-18T01:02:39.945 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_punch_hole": "false", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_ops": "50", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_seek_data_hole": "false", 2024-09-18T01:02:39.946 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc": "false", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_splice": "false", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_multiple": "2", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_rand_factor": "20", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_update_to": "1000", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_enable": "true", 2024-09-18T01:02:39.947 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_xfs_extsize": "false", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_zfs_snap": "false", 2024-09-18T01:02:39.948 INFO:teuthology.orchestra.run.smithi031.stdout: "fio_dir": "/tmp/fio", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_allow_other": "true", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_atomic_o_trunc": "true", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_big_writes": "true", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_debug": "false", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_default_permissions": "false", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_disable_pagecache": "false", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_max_write": "0", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_multithreaded": "true", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_require_active_mds": "true", 2024-09-18T01:02:39.949 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_set_user_groups": "true", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_move": "true", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_read": "true", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_write": "true", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_use_invalidate_cb": "true", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_target_name": "ceph", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_file": "", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_inject_failure": "0", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_interval": "5", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "host": "smithi031", 2024-09-18T01:02:39.950 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-18T01:02:39.951 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "inject_early_sigterm": "false", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_agent_port": "6799", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_tracing_enable": "false", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_aio": "true", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_align_min_size": "65536", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_align": "true", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_size": "4096", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_dio": "true", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_discard": "false", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_force_aio": "false", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_ignore_corruption": "false", 2024-09-18T01:02:39.952 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_bytes": "10485760", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_entries": "100", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_replay_from": "0", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_write_header_frequency": "0", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_zero_on_create": "false", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prefetch_periods": "10", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prezero_periods": "5", 2024-09-18T01:02:39.953 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_write_head_interval": "15", 2024-09-18T01:02:39.954 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "", 2024-09-18T01:02:39.954 INFO:teuthology.orchestra.run.smithi031.stdout: "keyfile": "", 2024-09-18T01:02:39.954 INFO:teuthology.orchestra.run.smithi031.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-18T01:02:39.954 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_backend": "rocksdb", 2024-09-18T01:02:39.954 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_default_stripe_size": "65536", 2024-09-18T01:02:39.954 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount": "false", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_bytes": "67108864", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_ops": "512", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_nid_prealloc": "1024", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_onode_map_size": "1024", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_submit_transaction": "false", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_transaction": "false", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "librados_thread_count": "2", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep": "false", 2024-09-18T01:02:39.955 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep_force_backtrace": "false", 2024-09-18T01:02:39.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_coarse_timestamps": "true", 2024-09-18T01:02:39.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-18T01:02:39.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_flush_on_exit": "false", 2024-09-18T01:02:39.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_host": "127.0.0.1", 2024-09-18T01:02:39.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_port": "12201", 2024-09-18T01:02:39.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_new": "1000", 2024-09-18T01:02:39.956 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_recent": "10000", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stderr_prefix": "", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stop_at_utilization": "0.970000", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": "true", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_graylog": "false", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_journald": "false", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_stderr": "false", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_syslog": "false", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max_rotating_auth_attempts": "10", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_action_on_write_error": "1", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_alternate_name_max": "8192", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_asio_thread_count": "2", 2024-09-18T01:02:39.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_export_pin": "true", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_dirs": "true", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_interval": "5", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_interval": "10", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max": "-1", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max_until": "-1", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_merge_size": "50", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_midchunk": "0.300000", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_start": "0.200000", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_minchunk": "0.001000", 2024-09-18T01:02:39.958 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_mode": "0", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_max": "1.200000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_min": "0.800000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_overload_epochs": "2", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_bits": "3", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_size": "10000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_target_decay": "10.000000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-18T01:02:39.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_grace": "15.000000", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_interval": "4.000000", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_mid": "0.700000", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_delay": "0", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_release_free_interval": "10", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_reservation": "0.050000", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-18T01:02:39.960 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_interval": "1", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_threshold": "262144", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_prealloc_inos": "1000", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_connect_bootstrapping": "false", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_damage_table_max_entries": "10000", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_auth_pins": "false", 2024-09-18T01:02:39.961 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_frag": "false", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_scatterstat": "false", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_subtrees": "false", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_decay_halflife": "5.000000", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_default_dir_hash": "2", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_defer_session_stale": "true", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_deny_all_reconnect": "false", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_keys_per_op": "16384", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_commit_size": "10", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_entries": "0", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_prefetch": "true", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-18T01:02:39.962 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_on_map": "false", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_early_reply": "true", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enable_op_tracker": "true", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enforce_unique_name": "true", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random": "true", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-18T01:02:39.963 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_summarize_threshold": "10", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_health_dummy": "false", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_migrator_session_race": "false", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-18T01:02:39.964 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_join_fs": "", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_journal_format": "1", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_create_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_export_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_import_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_expire_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_replay_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_link_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_mdstable_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_openc_at": "0", 2024-09-18T01:02:39.965 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_rename_at": "0", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_shutdown_at": "0", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_event_large_threshold": "524288", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_events_per_segment": "1024", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_events": "-1", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_segments": "128", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_pause": "false", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_segment_size": "0", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-18T01:02:39.966 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_threshold": "128", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_warn_factor": "2.000000", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_caps_per_client": "1048576", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_flushes": "100000", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_requests": "100000", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_export_size": "20971520", 2024-09-18T01:02:39.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_file_recover": "32", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_files": "64", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops": "8192", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_snaps_per_dir": "100", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_metrics_update_interval": "2", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_per_client": "100", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_working_set": "10000", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_numa_node": "-1", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-18T01:02:39.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_complaint_time": "30.000000", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_duration": "600", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_size": "20", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_size": "20", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_log_threshold": "5", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_grace": "15", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_interval": "5", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_caps": "30000", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-18T01:02:39.969 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_threshold": "262144", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_interval": "1.000000", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_gid": "0", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_uid": "0", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-18T01:02:39.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_shutdown_check": "0", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_skip_ino": "0", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-18T01:02:39.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_max_uid": "4294967294", 2024-09-18T01:02:39.972 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_min_uid": "0", 2024-09-18T01:02:39.972 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_rstat": "false", 2024-09-18T01:02:39.972 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_standby_replay_damaged": "false", 2024-09-18T01:02:39.972 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_symlink_recovery": "true", 2024-09-18T01:02:39.972 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-18T01:02:39.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_exports": "0", 2024-09-18T01:02:39.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_fragments": "0", 2024-09-18T01:02:39.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_tick_interval": "5.000000", 2024-09-18T01:02:39.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_valgrind_exit": "false", 2024-09-18T01:02:39.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_backtrace": "1", 2024-09-18T01:02:39.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_scatter": "false", 2024-09-18T01:02:39.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_sessions": "false", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool_debug": "false", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_device_bytes": "1073741824", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_set": "false", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_size": "65536", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_bytes": "134217728", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_messages": "512", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-18T01:02:39.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_disabled_modules": "", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_enable_op_tracker": "true", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_max_pg_num_change": "128", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_bytes": "134217728", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_messages": "128", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_bytes": "134217728", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_messages": "128", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_duration": "600", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_size": "20", 2024-09-18T01:02:39.975 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_log_threshold": "5", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_bytes": "536870912", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_messages": "8192", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_pool": "true", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_standby_modules": "true", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_period": "5", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_threshold": "5", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_tick_period": "2", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-18T01:02:39.976 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-18T01:02:39.977 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_delete": "true", 2024-09-18T01:02:39.977 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_size_one": "true", 2024-09-18T01:02:39.977 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_auth_validate_all_caps": "true", 2024-09-18T01:02:39.977 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-18T01:02:39.977 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-18T01:02:39.977 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_bytes": "104857600", 2024-09-18T01:02:39.977 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_directed_command_retry": "5", 2024-09-18T01:02:39.978 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-18T01:02:39.978 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-18T01:02:39.978 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-18T01:02:39.978 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-18T01:02:39.978 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_parallel": "3", 2024-09-18T01:02:39.978 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_log_interval": "1.000000", 2024-09-18T01:02:39.978 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_interval": "10.000000", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_target_rank": "-1", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file_level": "debug", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_file": "true", 2024-09-18T01:02:39.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_journald": "false", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_bootstrap": "false", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_start": "false", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_trim": "true", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-18T01:02:39.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cpu_threads": "4", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_crush_min_required_version": "hammer", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_daemon_bytes": "419430400", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_crit": "5", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_warn": "5", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_size_warn": "16106127360", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_json": "false", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-18T01:02:39.981 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_transactions": "false", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_extra_checks": "false", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_reef": "false", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_squid": "false", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_mkfs_grace": "120", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_uptime_grace": "60", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_default_strategy": "1", 2024-09-18T01:02:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_timeout": "5.000000", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_divisor": "2", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_enable_op_tracker": "true", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fake_pool_delete": "false", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_force_quorum_join": "false", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_globalid_prealloc": "10000", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_detail_to_clog": "false", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_log_update_period": "5", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_max_detail": "50", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog": "true", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_interval": "600", 2024-09-18T01:02:39.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host": "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.98", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host_override": "", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_initial_members": "", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease": "5.000000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-18T01:02:39.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_full_interval": "50", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max": "10000", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max_summary": "50", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_epochs": "500", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_osd": "10000", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pg_per_osd": "10000", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pool_pg_num": "65536", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-18T01:02:39.985 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_force_trim_to": "0", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_skip_sanity": "false", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_autotune": "true", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_target": "2147483648", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_beacon_grace": "30", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_digest_period": "5", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_inactive_grace": "60", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_min_osdmap_epochs": "500", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_complaint_time": "30", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_duration": "600", 2024-09-18T01:02:39.986 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_size": "20", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_size": "20", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_log_threshold": "5", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_in": "false", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-18T01:02:39.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-18T01:02:39.988 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size": "500", 2024-09-18T01:02:39.988 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-18T01:02:39.988 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-18T01:02:39.988 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-18T01:02:39.988 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_interval": "600", 2024-09-18T01:02:39.989 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-18T01:02:39.989 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-18T01:02:39.989 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_force_trim_to": "0", 2024-09-18T01:02:39.989 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-18T01:02:39.989 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_down_reporters": "2", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-18T01:02:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_report_timeout": "900", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_stuck_threshold": "60", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-18T01:02:39.991 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_probe_timeout": "2.000000", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_change": "0.050000", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_osds": "4", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-18T01:02:39.992 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_interval": "86400", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_max_keys": "100", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_timeout": "300", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_session_timeout": "300", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_smart_report_timeout": "5", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stat_smooth_intervals": "6", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_min_size": "2", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_size": "4", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_debug": "false", 2024-09-18T01:02:39.993 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_provider_kill_at": "0", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_requester_kill_at": "0", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_timeout": "60.000000", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_target_pg_per_osd": "100", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_tick_interval": "5", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_interval": "300.000000", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_use_min_delay_socket": "false", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_older_version_delay": "604800", 2024-09-18T01:02:39.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_misplaced": "false", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_older_version": "true", 2024-09-18T01:02:39.995 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "monmap": "", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-18T01:02:39.996 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_tracing_enabled": "false", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_op_threads": "3", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_cm": "false", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_device_name": "", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_dscp": "96", 2024-09-18T01:02:39.997 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_local_gid": "", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_port_num": "1", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_sl": "3", 2024-09-18T01:02:39.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_support_srq": "true", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_type": "ib", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_reap_threshold": "5", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_before_connect": "false", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv4": "true", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv6": "false", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr1": "true", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr2": "true", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_max": "7568", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_min": "6800", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_count": "3", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_delay": "5", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_client": "false", 2024-09-18T01:02:39.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mds": "false", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mgr": "false", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mon": "false", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_osd": "false", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_client_mode": "crc secure", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_mode": "crc secure", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_type": "", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_compress_secure": "false", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_idle_timeout": "900", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_ready_timeout": "10", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_data": "true", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_header": "true", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bad_msg": "false", 2024-09-18T01:02:40.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bug": "true", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_old_message": "true", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_skipped_message": "false", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_coremask": "0xF", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_enable_tso": "true", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hugepages": "", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-18T01:02:40.001 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_lro": "true", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_memory_channel": "4", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_pmd": "", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_port_id": "0", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_on_send": "false", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_initial_backoff": "0.200000", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_max": "1.000000", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_type": "", 2024-09-18T01:02:40.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_network_congestion": "0", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_socket_failures": "5000", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_learn_addr_from_peer": "true", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_accept_failures": "4", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_backoff": "15.000000", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_client_mode": "secure crc", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_service_mode": "secure crc", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_min_size": "1024", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_mode": "none", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-18T01:02:40.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_min_cost": "65536", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_public_type": "", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_service_mode": "crc secure", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_listen_backlog": "512", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_nodelay": "true", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_rcvbuf": "0", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_type": "async+posix", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "no_config_file": "false", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_completion_locks_per_session": "32", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_ops": "1024", 2024-09-18T01:02:40.004 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_tick_interval": "5.000000", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_timeout": "10.000000", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_blackhole": "false", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "openssl_engine_opts": "", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_delay_time": "5.000000", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_hist_halflife": "1000", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_low_ops": "2", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_ops": "4", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_slop": "0.020000", 2024-09-18T01:02:40.005 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_asio_thread_count": "2", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_async_recovery_min_cost": "100", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_max": "512", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_min": "64", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_degraded": "false", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_peering": "false", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_unfound": "true", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_beacon_report_interval": "300", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_duration": "30", 2024-09-18T01:02:40.006 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_max_block_size": "67108864", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blkin_trace_all": "false", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_for_log_corruption": "false", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T01:02:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_update_on_start": "true", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_cap": "256", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_size_cap": "524288000", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_op_priority": "63", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_watch_timeout": "30", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_command_max_records": "256", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_compact_on_start": "false", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_on_start": "true", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_weight_set": "true", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-18T01:02:40.008 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_feed_pullee": "-1", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_misdirected_ops": "false", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_acting_change": "false", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_purge_strays": "false", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_op_order": "false", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-18T01:02:40.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_shutdown": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_trim_objects": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_snaps": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_keys": "1024", 2024-09-18T01:02:40.010 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_stride": "524288", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_notify_timeout": "30", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep": "0.000000", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_discard_disconnected_ops": "true", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_enable_op_tracker": "true", 2024-09-18T01:02:40.011 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_info": "true", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown": "true", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_function_tracing": "false", 2024-09-18T01:02:40.012 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_grace": "20", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_interval": "6", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_peers": "10", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_size": "2000", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_stale": "600", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_max_size": "100000", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_min_size": "1000", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-18T01:02:40.013 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_size": "5120", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_kill_backfill_at": "0", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_cache_size": "50", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_dedup": "true", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max": "40", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_share_max_epochs": "40", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_name_len": "100", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_size": "0", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_backfills": "1", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_count": "5", 2024-09-18T01:02:40.014 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_period": "600", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_name_len": "2048", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_namespace_len": "256", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_size": "134217728", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_blocked_by": "16", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_log_entries": "10000", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pgls": "1024", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_cost": "8388608", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_objects": "10", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_scrubs": "3", 2024-09-18T01:02:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_trimming_pgs": "2", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_op_reply_len": "64", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_size": "90", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-18T01:02:40.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_profile": "balanced", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-18T01:02:40.017 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_base": "805306368", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_min": "134217728", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-18T01:02:40.018 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target": "4294967296", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_autotune": "false", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_pg_log_entries": "250", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_recovery_priority": "0", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_interval": "5", 2024-09-18T01:02:40.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-18T01:02:40.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-18T01:02:40.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_cache_shards": "32", 2024-09-18T01:02:40.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_op_tracker_shard": "32", 2024-09-18T01:02:40.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_auto_affinity": "true", 2024-09-18T01:02:40.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_node": "-1", 2024-09-18T01:02:40.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_prefer_iface": "true", 2024-09-18T01:02:40.020 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objecter_finishers": "1", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore": "bluestore", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_fuse": "false", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_tracing": "false", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_complaint_time": "30.000000", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_duration": "600", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_size": "20", 2024-09-18T01:02:40.021 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_size": "20", 2024-09-18T01:02:40.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-18T01:02:40.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_log_threshold": "5", 2024-09-18T01:02:40.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards": "0", 2024-09-18T01:02:40.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_hdd": "5", 2024-09-18T01:02:40.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_ssd": "8", 2024-09-18T01:02:40.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-18T01:02:40.022 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_min_cost": "65536", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue_cut_off": "high", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_timeout": "15", 2024-09-18T01:02:40.023 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_open_classes_on_start": "true", 2024-09-18T01:02:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_os_flags": "0", 2024-09-18T01:02:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_peering_op_priority": "255", 2024-09-18T01:02:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_cost": "1048576", 2024-09-18T01:02:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_priority": "5", 2024-09-18T01:02:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-18T01:02:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-18T01:02:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-18T01:02:40.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_max": "10000", 2024-09-18T01:02:40.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_min": "100", 2024-09-18T01:02:40.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-18T01:02:40.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-18T01:02:40.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-18T01:02:40.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-18T01:02:40.025 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-18T01:02:40.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-18T01:02:40.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-18T01:02:40.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-18T01:02:40.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-18T01:02:40.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-18T01:02:40.026 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crimson": "false", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-18T01:02:40.027 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flags": "0", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_min_size": "0", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_num": "32", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pgp_num": "0", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_ratio": "70", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_size": "2", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_type": "replicated", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_push_per_object_cost": "1000", 2024-09-18T01:02:40.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap": "true", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_cost": "20971520", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active": "0", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_single_start": "1", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_priority": "3", 2024-09-18T01:02:40.029 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_priority": "5", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep": "0.000000", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_repair_during_recovery": "false", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_requested_scrub_priority": "120", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair": "false", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-18T01:02:40.030 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_hour": "0", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_week_day": "0", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_max": "25", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_min": "5", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_cost": "52428800", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_during_recovery": "false", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_hour": "0", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_week_day": "0", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_event_cost": "4096", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_invalid_stats": "true", 2024-09-18T01:02:40.031 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_preemptions": "5", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_priority": "5", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_sleep": "0.000000", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_skip_data_digest": "false", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_smart_report_timeout": "5", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_cost": "1048576", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_priority": "5", 2024-09-18T01:02:40.032 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_transaction_size": "30", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-18T01:02:40.033 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tracing": "false", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_use_stale_snap": "false", 2024-09-18T01:02:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "osdc_blkin_trace_all": "false", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_kill_at": "0", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_max_join_drift": "10", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min": "500", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min_wait": "0.050000", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_propose_interval": "1.000000", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max": "500", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_min": "250", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_stash_full_interval": "25", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_max": "500", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_min": "250", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "perf": "true", 2024-09-18T01:02:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "-", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrv": "", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "public_bind_addr": "-", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network": "", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network_interface": "", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_busy_polling": "false", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_enabled": "false", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_session_max_number": "256", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_mon_op_timeout": "0", 2024-09-18T01:02:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_osd_op_timeout": "0", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_tracing": "false", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_atime_update_interval": "60", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_parent_reads": "false", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_snap_reads": "false", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blkin_trace_all": "false", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache": "true", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-18T01:02:40.037 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-18T01:02:40.040 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_policy": "writearound", 2024-09-18T01:02:40.040 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_size": "33554432", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_clone_copy_on_read": "false", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_compression_hint": "none", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_concurrent_management_ops": "10", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_clone_format": "auto", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_data_pool": "", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_features": "61", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_format": "2", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_map_options": "", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_order": "22", 2024-09-18T01:02:40.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_pool": "rbd", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_count": "0", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_unit": "0", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_enable_alloc_hint": "true", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler": "simple", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-18T01:02:40.042 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_order": "24", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_pool": "", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_splay_width": "4", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_parent_reads": "false", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_snap_reads": "false", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-18T01:02:40.043 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-18T01:02:40.044 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-18T01:02:40.045 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mtime_update_interval": "60", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_non_blocking_aio": "true", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_thread_timeout": "60", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_threads": "1", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_parent_cache_enabled": "false", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_plugins": "", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst": "0", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-18T01:02:40.046 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_limit": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_exclude_ops": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_limit": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-18T01:02:40.047 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_read_from_replica_policy": "default", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_skip_partial_discard": "true", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-18T01:02:40.048 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_tracing": "false", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_names": "true", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_pool": "true", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_base_url": "", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_log_level": "", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_acl_grants_max_num": "100", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_admin_entry": "admin", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_backend_store": "rados", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_barbican_url": "", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_beast_enable_async": "true", 2024-09-18T01:02:40.049 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache": "false", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_enabled": "true", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_expiry_interval": "900", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_lru_size": "25000", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_config_store": "rados", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_content_length_compat": "false", 2024-09-18T01:02:40.050 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress": "true", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cors_rules_max_num": "100", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cross_domain_policy": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_addr": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_password": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_username": "", 2024-09-18T01:02:40.051 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_require_ssl": "true", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-18T01:02:40.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_addr": "", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_auth": "token", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_namespace": "", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_prefix": "", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_token_file": "", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_buffersize": "524288", 2024-09-18T01:02:40.053 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_time": "300", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-18T01:02:40.054 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_port": "6379", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_changes_size": "1000", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_num_shards": "128", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_window": "30", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-18T01:02:40.055 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-18T01:02:40.056 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_name": "", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_s3website_name": "", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding": "true", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_gc_threads": "true", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_lc_threads": "true", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_ops_log": "false", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_quota_threads": "true", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_static_website": "false", 2024-09-18T01:02:40.057 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_usage_log": "false", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enforce_swift_acls": "true", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_exit_timeout_secs": "120", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_expose_bucket": "false", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_extended_http_attrs": "", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_filter": "none", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontends": "beast port=7480", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred": "50", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_objs": "32", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-18T01:02:40.058 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_period": "3600", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_init_timeout": "300", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_connection_idle": "300", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-18T01:02:40.059 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_domain": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password_path": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_project": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_tenant": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token_path": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_user": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_api_version": "2", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_domain": "", 2024-09-18T01:02:40.060 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_password": "", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_project": "", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_user": "", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_url": "", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_debug_interval": "-1", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_lock_max_time": "90", 2024-09-18T01:02:40.061 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_objs": "32", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_rules": "1000", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_worker": "3", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_thread_delay": "0", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_dnattr": "uid", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchfilter": "", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-18T01:02:40.062 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_uri": "", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_http_headers": "", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name_utc": "false", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_name_len": "0", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_size": "0", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-18T01:02:40.063 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_chunk_size": "4194304", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_listing_results": "1000", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_notify_retries": "10", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_param_size": "1048576", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_size": "5368709120", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_slo_entries": "1000", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_log_max_shards": "64", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-18T01:02:40.064 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mp_lock_max_time": "600", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_max_gc": "300", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-18T01:02:40.065 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_async_rados_threads": "32", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_control_oids": "8", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_numa_node": "-1", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_chunk_size": "100", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_gc_interval": "600", 2024-09-18T01:02:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_timeout": "600", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_tracing": "false", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_token": "", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_url": "", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_verify_ssl": "true", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-18T01:02:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_rados": "false", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_socket_path": "", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval": "2.000000", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lanes": "3", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-18T01:02:40.068 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_partitions": "3", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_continue": "true", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_tracing": "false", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm": "", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_id": "", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-18T01:02:40.069 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region": "", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_request_uri": "", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_batch_size": "64", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_max_aio": "128", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_num_logs": "16", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_thread_interval": "600", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_resolve_cname": "false", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_run_sync_thread": "true", 2024-09-18T01:02:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_scheduler_type": "throttler", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_script_uri": "", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_service_provider_name": "", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-18T01:02:40.071 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_id": "", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_secret": "", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_entry": "sts", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_key": "sts", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_min_session_duration": "900", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_token_introspection_url": "", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_account_in_url": "false", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_entry": "auth", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_url": "", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_custom_header": "", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_need_stats": "true", 2024-09-18T01:02:40.072 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_tenant_name": "", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_token_expiration": "86400", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url": "", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url_prefix": "swift", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_lease_period": "120", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-18T01:02:40.073 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_thread_pool_size": "512", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_comment": "", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_createby": "", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_encoding": "", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_flag": "false", 2024-09-18T01:02:40.074 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_origin": "", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_tracker": "", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_trust_forwarded_https": "false", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_shards": "32", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_user_shards": "1", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_use_opa_authz": "false", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache": "false", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-18T01:02:40.075 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_max_buckets": "1000", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_policies_max_num": "100", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_unique_email": "true", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_verify_ssl": "true", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone": "", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_id": "", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup": "", 2024-09-18T01:02:40.076 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_id": "", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_auto_session_clear": "true", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_block_size": "4096", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_size": "536870912", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-18T01:02:40.077 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_index_type": "binary_search", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_partition_filters": "false", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_perf": "false", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-18T01:02:40.078 INFO:teuthology.orchestra.run.smithi031.stdout: "run_dir": "/var/run/ceph", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_block_create": "true", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cache_lru_size": "67108864", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_max_object_size": "16777216", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_device_size": "53687091200", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_full_integrity_check": "false", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_capacity": "16", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-18T01:02:40.079 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_main_device_type": "SSD", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_segment_size": "67108864", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "set_keepcaps": "false", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "setgroup": "", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser": "", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser_match_path": "", 2024-09-18T01:02:40.080 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-18T01:02:40.081 INFO:teuthology.orchestra.run.smithi031.stdout: "thp": "false", 2024-09-18T01:02:40.081 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_default_timeout": "60", 2024-09-18T01:02:40.081 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-18T01:02:40.081 INFO:teuthology.orchestra.run.smithi031.stdout: "throttler_perf_counter": "true" 2024-09-18T01:02:40.081 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:40.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-18T01:02:40.198 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-18T01:02:40.198 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:02:40.208 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-18T01:02:40.208 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:02:40.217 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-18T01:02:40.217 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:02:40.228 DEBUG:teuthology.orchestra.run.smithi031:> type ceph-objectstore-tool 2024-09-18T01:02:40.245 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T01:02:40.246 DEBUG:teuthology.orchestra.run.smithi098:> type ceph-objectstore-tool 2024-09-18T01:02:40.264 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T01:02:40.264 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-18T01:02:40.274 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-18T01:02:40.274 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-18T01:02:40.274 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:02:40.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:02:40.275 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-18T01:02:40.275 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-18T01:02:40.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:02:40.276 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-18T01:02:40.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:40.276 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-18T01:02:40.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:02:40.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-18T01:02:40.288 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:40.286+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:40.339 INFO:teuthology.orchestra.run.smithi164.stdout:mapped writes DISABLED 2024-09-18T01:02:40.339 INFO:teuthology.orchestra.run.smithi164.stdout:mapped reads DISABLED 2024-09-18T01:02:40.339 INFO:teuthology.orchestra.run.smithi164.stdout:Seed set to 9499 2024-09-18T01:02:40.339 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-nbd mode enabled 2024-09-18T01:02:40.377 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:40.376+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:40.409 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:40.409 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:40.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:40.445 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:40.478 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:40.477+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:40.483 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:40.483 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:40.517 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:40.518 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:40.547 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:40.547 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:40.564 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:40.564 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:40.579 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:40.579 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:40.579 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:40.578+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:40.581 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:40.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:40.680 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:40.679+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:40.739 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:40.739 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:40.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:02:40.781 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:40.780+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:40.882 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:40.881+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:40.895 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:40.895 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:40.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:02:40.989 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:40.988+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:41.052 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:41.052 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:41.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:02:41.104 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:41.102+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:41.185 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:41.184+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:41.210 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:41.210 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:41.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:02:41.307 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:41.306+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:41.367 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:41.367 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:41.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:02:41.417 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:41.416+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:02:41.449 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:02:41.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:02:41.519 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:41.518+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:41.537 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:41.537 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:41.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:02:41.589 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:41.588+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:41.690 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:41.689+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:41.696 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:41.696 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:41.706 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:02:41.790 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:41.789+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:41.822 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:41.822 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:41.831 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:02:41.915 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:41.914+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:41.975 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:41.975 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:41.983 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:02:42.024 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:42.023+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:42.093 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:42.092+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:42.127 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:42.128 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:42.137 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:02:42.195 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:42.194+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:42.281 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:42.281 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:42.290 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:02:42.334 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:42.333+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:42.393 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:42.393 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:42.397 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:42.396+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:42.402 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:02:42.458 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:02:42.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:02:42.497 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:42.496+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:42.546 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:42.546 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:42.554 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:02:42.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:02:42.637 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:42.636+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:42.697 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:42.699 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:42.701 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:42.699+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:42.706 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:02:42.720 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:42.721 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:42.737 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:42.738 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:42.755 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:42.755 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:42.772 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:42.772 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:42.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:42.789 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:42.801 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:42.800+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:42.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:42.806 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:42.850 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:42.851 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:42.851 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:42.851 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:42.851 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:42.859 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:02:42.904 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:42.903+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:42.963 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:42.963 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:42.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:42.963 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:42.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 10.588223795999999, 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.964 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:42.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:02:42.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:02:42.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:02:42.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4208.0:1 2.3 e16/15 ondisk, result = 0)", 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.382210+0000", 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 10.579959374, 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000162571, 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:42.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382210+0000", 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382210+0000", 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382211+0000", 2024-09-18T01:02:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6600000000000005e-07 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382218+0000", 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3969999999999997e-06 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382230+0000", 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.13e-05 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382238+0000", 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0010000000000001e-06 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382275+0000", 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7163000000000003e-05 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382281+0000", 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4409999999999998e-06 2024-09-18T01:02:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382373+0000", 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1403000000000006e-05 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.384985+0000", 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 10.577184341000001, 2024-09-18T01:02:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014907700000000001, 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.384985+0000", 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.384985+0000", 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.384986+0000", 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.283e-06 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.384993+0000", 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.438e-06 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385006+0000", 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3504e-05 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385013+0000", 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.511e-06 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385052+0000", 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9119000000000003e-05 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385134+0000", 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2222e-05 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.385423+0000", 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 10.576746267000001, 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9093999999999997e-05, 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385423+0000", 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385423+0000", 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385424+0000", 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8100000000000002e-07 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385427+0000", 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9550000000000001e-06 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385435+0000", 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7970000000000001e-06 2024-09-18T01:02:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385439+0000", 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4399999999999998e-06 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385469+0000", 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9808999999999999e-05 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.385522+0000", 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3312000000000001e-05 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 9.5902124909999991, 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:02:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:02:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:02:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4208.0:5 2.2 e16/15 ondisk, result = 0)", 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.376855+0000", 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 9.5853144189999995, 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020918799999999999, 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376855+0000", 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376855+0000", 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376856+0000", 2024-09-18T01:02:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1399999999999995e-07 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376861+0000", 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8239999999999999e-06 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376870+0000", 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.3719999999999994e-06 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376879+0000", 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.8859999999999993e-06 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376957+0000", 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8343000000000005e-05 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376963+0000", 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0440000000000001e-06 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377064+0000", 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010080499999999999 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.5825355390000002, 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:02:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4208.0:7 2.2 e17/15 ondisk, result = 0)", 2024-09-18T01:02:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.383013+0000", 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.5791567630000003, 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.0059e-05, 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383013+0000", 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383013+0000", 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383014+0000", 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.424e-06 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383022+0000", 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3839999999999999e-06 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383034+0000", 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2738999999999999e-05 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383043+0000", 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1589999999999999e-06 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383088+0000", 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4663000000000003e-05 2024-09-18T01:02:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383094+0000", 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8449999999999999e-06 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383103+0000", 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.8449999999999992e-06 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:43.007 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:43.008 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:43.007+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:43.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:43.109 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:43.108+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:43.116 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:43.116 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:43.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:43.210 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:43.209+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:43.272 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:43.273 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:43.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:43.311 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:43.310+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:43.412 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:43.411+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:43.431 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:18 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'8, mlcod=13'5)", 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:26.320497+0000", 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.109045122000001, 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003496795, 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320497+0000", 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320497+0000", 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320498+0000", 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5799999999999998e-07 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320503+0000", 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320514+0000", 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320519+0000", 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3550000000000004e-06 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320668+0000", 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014891200000000001 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.320685+0000", 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6852000000000001e-05 2024-09-18T01:02:43.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.323952+0000", 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032673979999999999 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.323994+0000", 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1748000000000001e-05 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:55 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'33, mlcod=13'29)", 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:26.358618+0000", 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.070923824000001, 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003120425, 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358618+0000", 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358618+0000", 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358619+0000", 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5199999999999996e-07 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358692+0000", 2024-09-18T01:02:43.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2947999999999993e-05 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358700+0000", 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7160000000000003e-06 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358705+0000", 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-06 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358742+0000", 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6545000000000003e-05 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.358755+0000", 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3567000000000001e-05 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.361699+0000", 2024-09-18T01:02:43.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029438300000000001 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.361739+0000", 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9527000000000003e-05 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:56 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'34, mlcod=13'29)", 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:26.359196+0000", 2024-09-18T01:02:43.441 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.070346496999999, 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056577429999999998, 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359196+0000", 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359196+0000", 2024-09-18T01:02:43.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359196+0000", 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359202+0000", 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4210000000000003e-06 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359210+0000", 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2810000000000008e-06 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359216+0000", 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1149999999999999e-06 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359254+0000", 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7864999999999997e-05 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.359265+0000", 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1443999999999999e-05 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.364818+0000", 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055524700000000003 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.364853+0000", 2024-09-18T01:02:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5419000000000001e-05 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:58 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'36, mlcod=13'33)", 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:26.366150+0000", 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.063392089000001, 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037984529999999998, 2024-09-18T01:02:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.447 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366150+0000", 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366150+0000", 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366151+0000", 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7499999999999998e-07 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366216+0000", 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366229+0000", 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3176999999999999e-05 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366245+0000", 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5719e-05 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366292+0000", 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6888999999999998e-05 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366315+0000", 2024-09-18T01:02:43.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3898999999999999e-05 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.369914+0000", 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035986210000000002 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.369948+0000", 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4421000000000001e-05 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:59 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'37, mlcod=13'33)", 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:26.366579+0000", 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.062962999, 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033887029999999999, 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366579+0000", 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366579+0000", 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366580+0000", 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3e-07 2024-09-18T01:02:43.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366586+0000", 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7910000000000001e-06 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366596+0000", 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0489000000000001e-05 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366601+0000", 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0919999999999997e-06 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366648+0000", 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6743999999999998e-05 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.366662+0000", 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4185999999999999e-05 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.369951+0000", 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032885760000000001 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.369968+0000", 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7095000000000001e-05 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:109 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'71, mlcod=13'66)", 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:26.534231+0000", 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.895310908999999, 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032482969999999998, 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534231+0000", 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534231+0000", 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534232+0000", 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999998e-07 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534301+0000", 2024-09-18T01:02:43.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9394999999999995e-05 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534311+0000", 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0122000000000001e-05 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534316+0000", 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7210000000000002e-06 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534347+0000", 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0586999999999997e-05 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534357+0000", 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007e-05 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.537441+0000", 2024-09-18T01:02:43.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003084785 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.537479+0000", 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8099999999999998e-05 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:110 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'72, mlcod=13'68)", 2024-09-18T01:02:43.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:26.534875+0000", 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.894667154, 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031349889999999999, 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534875+0000", 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534875+0000", 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534875+0000", 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534885+0000", 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4010000000000003e-06 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534895+0000", 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0132999999999999e-05 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534901+0000", 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6459999999999998e-06 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534931+0000", 2024-09-18T01:02:43.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0233999999999999e-05 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.534942+0000", 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.128e-05 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.537972+0000", 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003029744 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:26.538010+0000", 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:130 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'88, mlcod=13'85)", 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.424862001000001, 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085906969999999996, 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004681+0000", 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5499999999999996e-07 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004687+0000", 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7459999999999997e-06 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004697+0000", 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0513000000000001e-05 2024-09-18T01:02:43.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004703+0000", 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004741+0000", 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7551e-05 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004751+0000", 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0497e-05 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013227+0000", 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084754370000000006 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013271+0000", 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4280000000000003e-05 2024-09-18T01:02:43.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.42433776, 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080847769999999996, 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005205+0000", 2024-09-18T01:02:43.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-07 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005281+0000", 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5927999999999995e-05 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005290+0000", 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5920000000000007e-06 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005295+0000", 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.605e-06 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005342+0000", 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7327000000000003e-05 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005364+0000", 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1583999999999999e-05 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013275+0000", 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007910594 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013289+0000", 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4561e-05 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.423802494, 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075705210000000002, 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3200000000000005e-07 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005746+0000", 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4090000000000002e-06 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005756+0000", 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0342e-05 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005761+0000", 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005814+0000", 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3603999999999997e-05 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005827+0000", 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2931e-05 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013291+0000", 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074636470000000003 2024-09-18T01:02:43.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013310+0000", 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9211999999999998e-05 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.006445+0000", 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.423096856000001, 2024-09-18T01:02:43.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068845750000000004, 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006445+0000", 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006445+0000", 2024-09-18T01:02:43.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006446+0000", 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3e-07 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006450+0000", 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4190000000000004e-06 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006460+0000", 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0162e-05 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006465+0000", 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006502+0000", 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6693999999999998e-05 2024-09-18T01:02:43.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006514+0000", 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1541e-05 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013314+0000", 2024-09-18T01:02:43.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067999519999999997 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013330+0000", 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6303999999999998e-05 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'86)", 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.006946+0000", 2024-09-18T01:02:43.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.422595990000001, 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00642867, 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006946+0000", 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006946+0000", 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006947+0000", 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006970+0000", 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3649e-05 2024-09-18T01:02:43.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006981+0000", 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0359e-05 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006985+0000", 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7060000000000003e-06 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007021+0000", 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5434e-05 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007032+0000", 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1416999999999999e-05 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013332+0000", 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062993370000000003 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013375+0000", 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3087999999999998e-05 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:135 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'93, mlcod=13'86)", 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.007492+0000", 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.422050531, 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059053150000000004, 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007492+0000", 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007492+0000", 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007492+0000", 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3e-07 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007498+0000", 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4809999999999997e-06 2024-09-18T01:02:43.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007508+0000", 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0229e-05 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007513+0000", 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6399999999999996e-06 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007548+0000", 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5821999999999997e-05 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007560+0000", 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1897e-05 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013377+0000", 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058164050000000002 2024-09-18T01:02:43.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013397+0000", 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0211e-05 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:136 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'94, mlcod=13'86)", 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.007838+0000", 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.421704404, 2024-09-18T01:02:43.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055769069999999999, 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007838+0000", 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007838+0000", 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007838+0000", 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.99e-07 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007843+0000", 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4769999999999997e-06 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007852+0000", 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9339999999999995e-06 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007856+0000", 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4209999999999997e-06 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007893+0000", 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999998e-05 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.007905+0000", 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1755e-05 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013398+0000", 2024-09-18T01:02:43.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054937479999999997 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013415+0000", 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6273000000000002e-05 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.008275+0000", 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.421267539999999, 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054708049999999996, 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008275+0000", 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008275+0000", 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008275+0000", 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6899999999999997e-07 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008316+0000", 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0936000000000002e-05 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008327+0000", 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0631e-05 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008333+0000", 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1739999999999997e-06 2024-09-18T01:02:43.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008381+0000", 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8195999999999997e-05 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008402+0000", 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0931999999999999e-05 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013725+0000", 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053228520000000003 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013745+0000", 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415e-05 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:138 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'96, mlcod=13'86)", 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.008741+0000", 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.420800708000002, 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050187610000000001, 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008741+0000", 2024-09-18T01:02:43.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008741+0000", 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008742+0000", 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6899999999999997e-07 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008746+0000", 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2529999999999998e-06 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008755+0000", 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3599999999999996e-06 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.008759+0000", 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4530000000000004e-06 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010361+0000", 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016021500000000001 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010372+0000", 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1141000000000001e-05 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013748+0000", 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033750989999999999 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013760+0000", 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2636000000000001e-05 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.009128+0000", 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.420414526999998, 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046659199999999996, 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009128+0000", 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009128+0000", 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009128+0000", 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5499999999999998e-07 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009133+0000", 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4490000000000001e-06 2024-09-18T01:02:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009143+0000", 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0441e-05 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009148+0000", 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010582+0000", 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001433871 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010602+0000", 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0659999999999999e-05 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013762+0000", 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031599280000000002 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013793+0000", 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1099999999999997e-05 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'86)", 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.420058457, 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069265170000000001, 2024-09-18T01:02:43.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0900000000000001e-07 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009528+0000", 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4047e-05 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009539+0000", 2024-09-18T01:02:43.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0444e-05 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009543+0000", 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6439999999999999e-06 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010886+0000", 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001342244 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010899+0000", 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2965e-05 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.016373+0000", 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054748009999999996 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.016410+0000", 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6662999999999998e-05 2024-09-18T01:02:43.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'86)", 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.009950+0000", 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.419592117000001, 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067388600000000002, 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009950+0000", 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009950+0000", 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009950+0000", 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009954+0000", 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3589999999999999e-06 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009961+0000", 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-18T01:02:43.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009965+0000", 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9469999999999996e-06 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013446+0000", 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034805669999999999 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013469+0000", 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3408999999999999e-05 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.016669+0000", 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00319996 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.016689+0000", 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9856000000000001e-05 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:161 1.0 e14/12 1:30306672:devicehealth::main.db.0000000000000000:head v 14'115, mlcod=14'112)", 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.449146+0000", 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.980395731, 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039798070000000001, 2024-09-18T01:02:43.525 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449146+0000", 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449146+0000", 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449147+0000", 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449209+0000", 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2093000000000003e-05 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449216+0000", 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4159999999999998e-06 2024-09-18T01:02:43.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449220+0000", 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3900000000000002e-06 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449250+0000", 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9964999999999999e-05 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.449261+0000", 2024-09-18T01:02:43.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1651000000000001e-05 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.453096+0000", 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003834432 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.453126+0000", 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0394000000000001e-05 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.530 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:43.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:02:43.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:02:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:02:43.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:02:43.534 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:43.531+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:43.634 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:43.632+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:43.687 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:43.687 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:43.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:02:43.741 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:43.740+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:43.805 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:43.805 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:43.805 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:43.805 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:43.805 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:43.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:02:43.834 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:43.833+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:43.935 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:43.934+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:43.962 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:43.962 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:43.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:43.962 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:43.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4208.0:5 2.2 e17/15 2:5cea7035:::rbd_info:head v 17'1, mlcod=0'0)", 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:33.375663+0000", 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.585968355, 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001090494, 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375663+0000", 2024-09-18T01:02:43.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375663+0000", 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375663+0000", 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.22e-07 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375670+0000", 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3080000000000004e-06 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375686+0000", 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6504999999999999e-05 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375697+0000", 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0305999999999999e-05 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375768+0000", 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1352000000000004e-05 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.375812+0000", 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3735000000000001e-05 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.376715+0000", 2024-09-18T01:02:43.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00090303499999999995 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:33.376754+0000", 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9031e-05 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4208.0:7 2.2 e18/15 2:5cea7035:::rbd_info:head v 18'2, mlcod=0'0)", 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:34.381949+0000", 2024-09-18T01:02:43.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5796828979999997, 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001030963, 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.381949+0000", 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.381949+0000", 2024-09-18T01:02:43.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.381949+0000", 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7500000000000001e-07 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.381957+0000", 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3289999999999994e-06 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.381974+0000", 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6565000000000001e-05 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.381988+0000", 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4311000000000001e-05 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.382061+0000", 2024-09-18T01:02:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3281000000000005e-05 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.382083+0000", 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1270000000000001e-05 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.382920+0000", 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083700300000000001 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:34.382980+0000", 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9929000000000002e-05 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4355.0:9 3.1a e23/20 3:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:43.473940+0000", 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.48769122199999998, 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00090102599999999998, 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.972 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.473940+0000", 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.473940+0000", 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.473941+0000", 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.473947+0000", 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7710000000000001e-06 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.473960+0000", 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2395000000000001e-05 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.473972+0000", 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.221e-05 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.474040+0000", 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8067999999999999e-05 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.474066+0000", 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5751000000000001e-05 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.474790+0000", 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072423099999999999 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.474841+0000", 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1214999999999997e-05 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:13 3.19 3:9d537db2:::rbd_object_map.110387a2bfdc:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:43.490371+0000", 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47126018200000003, 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001253728, 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490371+0000", 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490371+0000", 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490373+0000", 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.299e-06 2024-09-18T01:02:43.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490378+0000", 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-06 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490387+0000", 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8850000000000005e-06 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490395+0000", 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9789999999999993e-06 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490444+0000", 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8764999999999998e-05 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490525+0000", 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1379999999999997e-05 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.490652+0000", 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012659300000000001 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491286+0000", 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063421099999999998 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491560+0000", 2024-09-18T01:02:43.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027356800000000002 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491596+0000", 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5926000000000001e-05 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491625+0000", 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9431999999999999e-05 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4355.0:13 3.19 e23/20 ondisk, result = 0)", 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:43.491486+0000", 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47014598499999999, 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000145344, 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491486+0000", 2024-09-18T01:02:43.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491486+0000", 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491486+0000", 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:43.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491491+0000", 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5589999999999999e-06 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491500+0000", 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1770000000000004e-06 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491512+0000", 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2122e-05 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491550+0000", 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8207999999999997e-05 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491557+0000", 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1519999999999997e-06 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:43.491631+0000", 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4312000000000003e-05 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:43.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:43.987 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:43.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:02:44.072 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:44.071+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:44.136 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:44.136 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:44.137 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:44.137+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:44.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:02:44.239 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:44.238+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:44.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:02:44.343 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:44.342+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:44.441 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:44.440+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4208.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.070994295, 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00414577, 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6500000000000001e-07 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378024+0000", 2024-09-18T01:02:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7300000000000005e-06 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378036+0000", 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1809e-05 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378045+0000", 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2770000000000005e-06 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378085+0000", 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0194000000000002e-05 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378105+0000", 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0693999999999999e-05 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.382107+0000", 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040012729999999996 2024-09-18T01:02:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.382162+0000", 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5627999999999997e-05 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:44.454 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:44.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:02:44.463 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:02:44.466 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:02:44.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:02:44.541 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:44.540+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:44.573 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:44.573 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:44.573 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:02:44.574 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:44.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:02:44.642 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:44.641+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:44.725 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:44.725 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:44.725 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:44.725 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:44.725 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:44.734 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:02:44.777 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:44.776+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:44.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:02:44.838 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.498258849999999, 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0076954079999999999, 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.839 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 37 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:44.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337902+0000", 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6599999999999999e-06 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337905+0000", 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.371e-06 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337913+0000", 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1580000000000005e-06 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341291+0000", 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033772979999999999 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341344+0000", 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3483999999999998e-05 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341426+0000", 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1462e-05 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.343020+0000", 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015945740000000001 2024-09-18T01:02:44.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345556+0000", 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025358749999999999 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345578+0000", 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1865000000000002e-05 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345594+0000", 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6430999999999999e-05 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.304251269000002, 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077811820000000002, 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.845 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 111 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531911+0000", 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4929999999999998e-06 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531917+0000", 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5509999999999996e-06 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531934+0000", 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6356000000000001e-05 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535192+0000", 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032581369999999999 2024-09-18T01:02:44.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535241+0000", 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9363999999999997e-05 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535318+0000", 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7078999999999998e-05 2024-09-18T01:02:44.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538191+0000", 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0028732240000000002 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539619+0000", 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014276619999999999 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539661+0000", 2024-09-18T01:02:44.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1819000000000001e-05 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539688+0000", 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7030999999999999e-05 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:44.851 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.304209104000002, 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077932879999999998, 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 112 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:44.852 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6100000000000001e-07 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532030+0000", 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1082999999999996e-05 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532040+0000", 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9029999999999999e-06 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532047+0000", 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535639+0000", 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0035927009999999998 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535685+0000", 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5661000000000002e-05 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535757+0000", 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2113999999999994e-05 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538221+0000", 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024644440000000001 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539694+0000", 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001472283 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539719+0000", 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4777999999999999e-05 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539742+0000", 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3547e-05 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.857 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.304098070999999, 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077280339999999999, 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.858 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 113 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532061+0000", 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1900000000000002e-07 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532064+0000", 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8720000000000004e-06 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532078+0000", 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3766e-05 2024-09-18T01:02:44.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532085+0000", 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.939e-06 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536066+0000", 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039811229999999996 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536113+0000", 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6292000000000003e-05 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536186+0000", 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3632000000000002e-05 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538252+0000", 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020659419999999999 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539746+0000", 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014934670000000001 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539770+0000", 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4311000000000002e-05 2024-09-18T01:02:44.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539788+0000", 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7971000000000001e-05 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.304061675, 2024-09-18T01:02:44.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.007737715, 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 114 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:44.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532097+0000", 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3499999999999996e-07 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532102+0000", 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8770000000000001e-06 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532108+0000", 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.117e-06 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532118+0000", 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0101999999999999e-05 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536401+0000", 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042836200000000001 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536448+0000", 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.638e-05 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536522+0000", 2024-09-18T01:02:44.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3769000000000005e-05 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538275+0000", 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017533819999999999 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539792+0000", 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001516861 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539814+0000", 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2305000000000001e-05 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539834+0000", 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9766999999999999e-05 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.870 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.304019836999998, 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010209316, 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 115 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:02:44.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532186+0000", 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7521999999999997e-05 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532192+0000", 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5049999999999998e-06 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532195+0000", 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5159999999999999e-06 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536736+0000", 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0045408999999999996 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536783+0000", 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6956999999999998e-05 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536851+0000", 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538944+0000", 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020932110000000002 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542279+0000", 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033348679999999999 2024-09-18T01:02:44.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542321+0000", 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1520999999999998e-05 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542347+0000", 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6789000000000001e-05 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.303955844000001, 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010197308, 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 116 2024-09-18T01:02:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532203+0000", 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532205+0000", 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5610000000000001e-06 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532209+0000", 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9639999999999997e-06 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532213+0000", 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5939999999999998e-06 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537234+0000", 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050209440000000003 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537281+0000", 2024-09-18T01:02:44.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6955e-05 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537352+0000", 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1870000000000001e-05 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538964+0000", 2024-09-18T01:02:44.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001611517 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542353+0000", 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003389076 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542376+0000", 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3456e-05 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542399+0000", 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2935e-05 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.303940742000002, 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010229594999999999, 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.883 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 117 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.884 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.03e-07 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532220+0000", 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1710000000000001e-06 2024-09-18T01:02:44.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532223+0000", 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4740000000000001e-06 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532227+0000", 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5920000000000001e-06 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537567+0000", 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053407589999999996 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537615+0000", 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7590999999999999e-05 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537694+0000", 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9015000000000002e-05 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538995+0000", 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013009429999999999 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542403+0000", 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034077180000000001 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542428+0000", 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5607000000000001e-05 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542447+0000", 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8422000000000001e-05 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.832210266000001, 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0095908409999999993, 2024-09-18T01:02:44.889 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 130 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0500000000000002e-07 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003953+0000", 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2860000000000001e-06 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003964+0000", 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1259e-05 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003970+0000", 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1349999999999997e-06 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004461+0000", 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00049077799999999998 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004507+0000", 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5677000000000003e-05 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004590+0000", 2024-09-18T01:02:44.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3610999999999996e-05 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006628+0000", 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020381240000000001 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013439+0000", 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.006810633 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013498+0000", 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9101999999999998e-05 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013538+0000", 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0431e-05 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:44.895 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.832172019000001, 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096617260000000007, 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 131 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:44.896 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003987+0000", 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004027+0000", 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0516999999999998e-05 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004034+0000", 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5220000000000004e-06 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004038+0000", 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0960000000000003e-06 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004894+0000", 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00085607399999999996 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004982+0000", 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7811000000000003e-05 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005115+0000", 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013318600000000001 2024-09-18T01:02:44.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006634+0000", 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001519643 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013579+0000", 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0069450140000000002 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013610+0000", 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0849999999999998e-05 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013648+0000", 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7372000000000001e-05 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.901 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.832113263, 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096891579999999998, 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 132 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.902 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.15e-07 2024-09-18T01:02:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004047+0000", 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.232e-06 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004051+0000", 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.58e-06 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.904 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004056+0000", 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0839999999999999e-06 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005528+0000", 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014725350000000001 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005573+0000", 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4518999999999998e-05 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005648+0000", 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4757000000000003e-05 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007604+0000", 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019566739999999998 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013684+0000", 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0060797519999999999 2024-09-18T01:02:44.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013712+0000", 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8214000000000001e-05 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013734+0000", 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1495999999999999e-05 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831990310999998, 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096596410000000001, 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 133 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.908 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:44.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4099999999999998e-07 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004173+0000", 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4270000000000001e-06 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004188+0000", 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5435999999999999e-05 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004198+0000", 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0034000000000001e-05 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006144+0000", 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019458489999999999 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.911 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006225+0000", 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1207999999999999e-05 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006346+0000", 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012035400000000001 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008493+0000", 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021473849999999999 2024-09-18T01:02:44.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013769+0000", 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052760510000000004 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013795+0000", 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6205000000000001e-05 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.913 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013827+0000", 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1951e-05 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831944691, 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097221809999999999, 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.914 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 134 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004214+0000", 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004280+0000", 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5707e-05 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.916 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004291+0000", 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.154e-05 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004306+0000", 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5088e-05 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006681+0000", 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002374953 2024-09-18T01:02:44.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006754+0000", 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2490000000000006e-05 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006825+0000", 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1667000000000003e-05 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009234+0000", 2024-09-18T01:02:44.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002408477 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013872+0000", 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0046381599999999997 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013906+0000", 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3878999999999999e-05 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013935+0000", 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9598e-05 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831835948999998, 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097082049999999993, 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.920 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 135 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004323+0000", 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.922 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004328+0000", 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5559999999999998e-06 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004338+0000", 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9699999999999994e-06 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004344+0000", 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4009999999999996e-06 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007275+0000", 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002931309 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007321+0000", 2024-09-18T01:02:44.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6292000000000003e-05 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007394+0000", 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2354000000000006e-05 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010084+0000", 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0026903740000000002 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013970+0000", 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0038859210000000001 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013997+0000", 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.669e-05 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014030+0000", 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.345e-05 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.83180462, 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097521970000000006, 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.927 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 136 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004354+0000", 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7800000000000001e-07 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004360+0000", 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9660000000000001e-06 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004370+0000", 2024-09-18T01:02:44.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0550999999999999e-05 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004374+0000", 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0500000000000002e-06 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007634+0000", 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003259498 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.930 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007684+0000", 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9768000000000002e-05 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007758+0000", 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4017999999999995e-05 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010088+0000", 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0023301569999999998 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014061+0000", 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039732439999999999 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014087+0000", 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.618e-05 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014105+0000", 2024-09-18T01:02:44.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8187000000000001e-05 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831773435999999, 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098164900000000006, 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.933 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 137 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:44.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004385+0000", 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004432+0000", 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7184000000000001e-05 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004437+0000", 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1730000000000003e-06 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004444+0000", 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008052+0000", 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036079279999999998 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008097+0000", 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5011e-05 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008170+0000", 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2904999999999994e-05 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010090+0000", 2024-09-18T01:02:44.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001920274 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014138+0000", 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040482019999999999 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014166+0000", 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7657999999999998e-05 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014201+0000", 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4888000000000002e-05 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831701316, 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098169759999999998, 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.939 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 138 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004460+0000", 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2499999999999998e-06 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004464+0000", 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3239999999999996e-06 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004467+0000", 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8289999999999999e-06 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008528+0000", 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040605679999999996 2024-09-18T01:02:44.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008574+0000", 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6492999999999997e-05 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008645+0000", 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0403999999999999e-05 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010093+0000", 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001447802 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014230+0000", 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0041371280000000003 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014254+0000", 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4757e-05 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014274+0000", 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9105000000000001e-05 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831686326, 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098886330000000008, 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.945 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 139 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004475+0000", 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8720000000000002e-06 2024-09-18T01:02:44.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004479+0000", 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1060000000000002e-06 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004482+0000", 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7369999999999998e-06 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008917+0000", 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.004435184 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008966+0000", 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9592000000000002e-05 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009037+0000", 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0797999999999995e-05 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010095+0000", 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0010581399999999999 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014302+0000", 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042062189999999998 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014328+0000", 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6432e-05 2024-09-18T01:02:44.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014360+0000", 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2190000000000002e-05 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.951 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831672733000001, 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.012201405, 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 140 2024-09-18T01:02:44.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-07 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004529+0000", 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3432e-05 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004533+0000", 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5249999999999999e-06 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004537+0000", 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8160000000000004e-06 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009263+0000", 2024-09-18T01:02:44.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0047261509999999996 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009311+0000", 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7568000000000003e-05 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009381+0000", 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0419000000000005e-05 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010098+0000", 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00071670299999999996 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016625+0000", 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0065267939999999998 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016651+0000", 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5959999999999999e-05 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016687+0000", 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5772000000000002e-05 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.831615340999999, 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.012284938, 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:44.959 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 141 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2300000000000002e-07 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004546+0000", 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.864e-06 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004552+0000", 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1999999999999999e-06 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004555+0000", 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.706e-06 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009757+0000", 2024-09-18T01:02:44.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052024180000000003 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009806+0000", 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8653999999999999e-05 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009879+0000", 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3551000000000001e-05 2024-09-18T01:02:44.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010335+0000", 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00045522899999999997 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016767+0000", 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0064324079999999997 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016790+0000", 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3468999999999998e-05 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016828+0000", 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7116000000000002e-05 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.965 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:44.966 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:44.966 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:44.967 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:44.967 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:44.968 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:44.968 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:44.969 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:02:44.969 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:44.966+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:44.973 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:44.973 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:44.991 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:44.991 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:45.007 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:45.008 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:45.024 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:45.025 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:45.067 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:45.066+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:45.111 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.112 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.120 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:02:45.168 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:45.167+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:45.223 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.223 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.232 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:02:45.269 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:45.268+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:45.336 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.336 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.345 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:02:45.365 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero length punch hole 2024-09-18T01:02:45.365 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero length punch hole 2024-09-18T01:02:45.369 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:45.368+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:45.392 INFO:teuthology.orchestra.run.smithi164.stdout:9 write 0x7f31000 thru 0x7f3eeec (0xdeed bytes) 2024-09-18T01:02:45.392 INFO:teuthology.orchestra.run.smithi164.stdout:10 read 0x4b9f600 thru 0x4bae893 (0xf294 bytes) 2024-09-18T01:02:45.392 INFO:teuthology.orchestra.run.smithi164.stdout:11 read 0x42fe000 thru 0x430270e (0x470f bytes) 2024-09-18T01:02:45.393 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0x32e1400 2024-09-18T01:02:45.397 INFO:teuthology.orchestra.run.smithi164.stdout:13 trunc from 0x7f3f000 to 0x32e1400 2024-09-18T01:02:45.425 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0x484cc00 2024-09-18T01:02:45.426 INFO:teuthology.orchestra.run.smithi164.stdout:14 trunc from 0x32e1400 to 0x484cc00 2024-09-18T01:02:45.435 INFO:teuthology.orchestra.run.smithi164.stdout:17 read 0x2fc5600 thru 0x2fd3e5d (0xe85e bytes) 2024-09-18T01:02:45.436 INFO:teuthology.orchestra.run.smithi164.stdout:18 read 0x2c4e600 thru 0x2c4ed6e (0x76f bytes) 2024-09-18T01:02:45.455 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.455 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.456 INFO:teuthology.orchestra.run.smithi164.stdout:19 write 0xbb98200 thru 0xbba699a (0xe79b bytes) 2024-09-18T01:02:45.457 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xd25bc00 2024-09-18T01:02:45.459 INFO:teuthology.orchestra.run.smithi164.stdout:23 trunc from 0xbba6a00 to 0xd25bc00 2024-09-18T01:02:45.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:02:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:02:45.470 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:45.469+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:45.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:02:45.493 INFO:teuthology.orchestra.run.smithi164.stdout:24 write 0xed45c00 thru 0xed4db8d (0x7f8e bytes) 2024-09-18T01:02:45.493 INFO:teuthology.orchestra.run.smithi164.stdout:25 read 0x4575600 thru 0x457eb5e (0x955f bytes) 2024-09-18T01:02:45.493 INFO:teuthology.orchestra.run.smithi164.stdout:27 read 0xd3e2200 thru 0xd3ec67b (0xa47c bytes) 2024-09-18T01:02:45.493 INFO:teuthology.orchestra.run.smithi164.stdout:30 read 0x1d68c00 thru 0x1d68f1c (0x31d bytes) 2024-09-18T01:02:45.500 INFO:teuthology.orchestra.run.smithi164.stdout:33 trunc from 0xed4dc00 to 0x8226600 2024-09-18T01:02:45.527 INFO:teuthology.orchestra.run.smithi164.stdout:34 trunc from 0x8226600 to 0x5497a00 2024-09-18T01:02:45.568 INFO:teuthology.orchestra.run.smithi164.stdout:35 write 0x963fc00 thru 0x964509b (0x549c bytes) 2024-09-18T01:02:45.568 INFO:teuthology.orchestra.run.smithi164.stdout:38 punch from 0x57f7400 to 0x57f9c00, (0x2800 bytes) 2024-09-18T01:02:45.571 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:45.570+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:45.578 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.578 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.588 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:02:45.589 INFO:teuthology.orchestra.run.smithi164.stdout:41 write 0xca37e00 thru 0xca3d1de (0x53df bytes) 2024-09-18T01:02:45.589 INFO:teuthology.orchestra.run.smithi164.stdout:42 read 0x3afd200 thru 0x3afda11 (0x812 bytes) 2024-09-18T01:02:45.589 INFO:teuthology.orchestra.run.smithi164.stdout:48 read 0x9e9d200 thru 0x9ea1630 (0x4431 bytes) 2024-09-18T01:02:45.589 INFO:teuthology.orchestra.run.smithi164.stdout:50 read 0xac3da00 thru 0xac43852 (0x5e53 bytes) 2024-09-18T01:02:45.612 INFO:teuthology.orchestra.run.smithi164.stdout:54 write 0xed34000 thru 0xed40583 (0xc584 bytes) 2024-09-18T01:02:45.612 INFO:teuthology.orchestra.run.smithi164.stdout:55 punch from 0xa472c00 to 0xa477a00, (0x4e00 bytes) 2024-09-18T01:02:45.612 INFO:teuthology.orchestra.run.smithi164.stdout:56 write 0x7a21200 thru 0x7a22bec (0x19ed bytes) 2024-09-18T01:02:45.612 INFO:teuthology.orchestra.run.smithi164.stdout:58 punch from 0x392be00 to 0x3934a00, (0x8c00 bytes) 2024-09-18T01:02:45.612 INFO:teuthology.orchestra.run.smithi164.stdout:60 write 0x6d8d800 thru 0x6d9d4c8 (0xfcc9 bytes) 2024-09-18T01:02:45.613 INFO:teuthology.orchestra.run.smithi164.stdout:63 read 0x861d600 thru 0x862638d (0x8d8e bytes) 2024-09-18T01:02:45.613 INFO:teuthology.orchestra.run.smithi164.stdout:65 read 0x40aa000 thru 0x40ac039 (0x203a bytes) 2024-09-18T01:02:45.613 INFO:teuthology.orchestra.run.smithi164.stdout:67 write 0xd872a00 thru 0xd875189 (0x278a bytes) 2024-09-18T01:02:45.613 INFO:teuthology.orchestra.run.smithi164.stdout:68 write 0xe9eb400 thru 0xe9f4b59 (0x975a bytes) 2024-09-18T01:02:45.613 INFO:teuthology.orchestra.run.smithi164.stdout:71 punch from 0xe348a00 to 0xe348e00, (0x400 bytes) 2024-09-18T01:02:45.614 INFO:teuthology.orchestra.run.smithi164.stdout:72 punch from 0x83f9e00 to 0x83fba00, (0x1c00 bytes) 2024-09-18T01:02:45.614 INFO:teuthology.orchestra.run.smithi164.stdout:74 punch from 0x8784600 to 0x878f400, (0xae00 bytes) 2024-09-18T01:02:45.614 INFO:teuthology.orchestra.run.smithi164.stdout:77 punch from 0xcaae200 to 0xcab7c00, (0x9a00 bytes) 2024-09-18T01:02:45.614 INFO:teuthology.orchestra.run.smithi164.stdout:78 write 0xe952400 thru 0xe959772 (0x7373 bytes) 2024-09-18T01:02:45.614 INFO:teuthology.orchestra.run.smithi164.stdout:79 read 0xc718c00 thru 0xc725750 (0xcb51 bytes) 2024-09-18T01:02:45.620 INFO:teuthology.orchestra.run.smithi164.stdout:80 trunc from 0xed40600 to 0x8b64600 2024-09-18T01:02:45.665 INFO:teuthology.orchestra.run.smithi164.stdout:82 write 0xcab2800 thru 0xcabefac (0xc7ad bytes) 2024-09-18T01:02:45.672 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:45.671+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:45.680 INFO:teuthology.orchestra.run.smithi164.stdout:85 write 0xebcd200 thru 0xebdd172 (0xff73 bytes) 2024-09-18T01:02:45.681 INFO:teuthology.orchestra.run.smithi164.stdout:86 read 0xa2b6e00 thru 0xa2c1912 (0xab13 bytes) 2024-09-18T01:02:45.681 INFO:teuthology.orchestra.run.smithi164.stdout:89 read 0x5ff0200 thru 0x5ffc5a6 (0xc3a7 bytes) 2024-09-18T01:02:45.681 INFO:teuthology.orchestra.run.smithi164.stdout:92 read 0x7f66e00 thru 0x7f687a9 (0x19aa bytes) 2024-09-18T01:02:45.681 INFO:teuthology.orchestra.run.smithi164.stdout:95 read 0xb8fe200 thru 0xb902caa (0x4aab bytes) 2024-09-18T01:02:45.696 INFO:teuthology.orchestra.run.smithi164.stdout:96 trunc from 0xebdd200 to 0x2019600 2024-09-18T01:02:45.741 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.741 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.750 INFO:teuthology.orchestra.run.smithi164.stdout:99 write 0xee17400 thru 0xee1827e (0xe7f bytes) 2024-09-18T01:02:45.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:45.763 INFO:teuthology.orchestra.run.smithi164.stdout:101 trunc from 0xee18400 to 0x36d8a00 2024-09-18T01:02:45.773 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:45.772+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:45.794 INFO:teuthology.orchestra.run.smithi164.stdout:102 read 0x591600 thru 0x591eb2 (0x8b3 bytes) 2024-09-18T01:02:45.816 INFO:teuthology.orchestra.run.smithi164.stdout:103 write 0x8b44400 thru 0x8b4e7c3 (0xa3c4 bytes) 2024-09-18T01:02:45.816 INFO:teuthology.orchestra.run.smithi164.stdout:104 write 0x3ef5200 thru 0x3f016f2 (0xc4f3 bytes) 2024-09-18T01:02:45.816 INFO:teuthology.orchestra.run.smithi164.stdout:108 write 0x60f7400 thru 0x61017e7 (0xa3e8 bytes) 2024-09-18T01:02:45.817 INFO:teuthology.orchestra.run.smithi164.stdout:109 punch from 0x53f4e00 to 0x5400a00, (0xbc00 bytes) 2024-09-18T01:02:45.825 INFO:teuthology.orchestra.run.smithi164.stdout:110 trunc from 0x8b4e800 to 0x11f9e00 2024-09-18T01:02:45.852 INFO:teuthology.orchestra.run.smithi164.stdout:111 trunc from 0x11f9e00 to 0x1bc1c00 2024-09-18T01:02:45.864 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.864 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:45.874 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:45.873+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:45.886 INFO:teuthology.orchestra.run.smithi164.stdout:113 write 0x40c4000 thru 0x40ca723 (0x6724 bytes) 2024-09-18T01:02:45.887 INFO:teuthology.orchestra.run.smithi164.stdout:114 read 0x2d48400 thru 0x2d4c546 (0x4147 bytes) 2024-09-18T01:02:45.887 INFO:teuthology.orchestra.run.smithi164.stdout:118 read 0x6d4400 thru 0x6e2619 (0xe21a bytes) 2024-09-18T01:02:45.887 INFO:teuthology.orchestra.run.smithi164.stdout:120 read 0x1cc2e00 thru 0x1ccd972 (0xab73 bytes) 2024-09-18T01:02:45.887 INFO:teuthology.orchestra.run.smithi164.stdout:122 read 0x282da00 thru 0x2833a9d (0x609e bytes) 2024-09-18T01:02:45.914 INFO:teuthology.orchestra.run.smithi164.stdout:124 write 0x8197c00 thru 0x81a2698 (0xaa99 bytes) 2024-09-18T01:02:45.914 INFO:teuthology.orchestra.run.smithi164.stdout:125 write 0x110b400 thru 0x11192cf (0xded0 bytes) 2024-09-18T01:02:45.914 INFO:teuthology.orchestra.run.smithi164.stdout:127 write 0x5cbe800 thru 0x5cc6dc0 (0x85c1 bytes) 2024-09-18T01:02:45.914 INFO:teuthology.orchestra.run.smithi164.stdout:129 read 0x5e62a00 thru 0x5e6f675 (0xcc76 bytes) 2024-09-18T01:02:45.914 INFO:teuthology.orchestra.run.smithi164.stdout:130 read 0xbf9a00 thru 0xc023d3 (0x89d4 bytes) 2024-09-18T01:02:45.914 INFO:teuthology.orchestra.run.smithi164.stdout:131 read 0xee3200 thru 0xeea734 (0x7535 bytes) 2024-09-18T01:02:45.918 INFO:teuthology.orchestra.run.smithi164.stdout:133 trunc from 0x81a2800 to 0x4643600 2024-09-18T01:02:45.970 INFO:teuthology.orchestra.run.smithi164.stdout:134 write 0xec92600 thru 0xec99da5 (0x77a6 bytes) 2024-09-18T01:02:45.970 INFO:teuthology.orchestra.run.smithi164.stdout:137 read 0x2b86e00 thru 0x2b89b4c (0x2d4d bytes) 2024-09-18T01:02:45.973 INFO:teuthology.orchestra.run.smithi164.stdout:138 trunc from 0xec99e00 to 0xbebfc00 2024-09-18T01:02:45.974 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:45.973+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:45.985 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:45.985 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:45.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:46.003 INFO:teuthology.orchestra.run.smithi164.stdout:140 trunc from 0xbebfc00 to 0x8bf0200 2024-09-18T01:02:46.021 INFO:teuthology.orchestra.run.smithi164.stdout:141 trunc from 0x8bf0200 to 0x6b44600 2024-09-18T01:02:46.057 INFO:teuthology.orchestra.run.smithi164.stdout:142 write 0xbf95400 thru 0xbf96acf (0x16d0 bytes) 2024-09-18T01:02:46.075 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:46.074+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:46.076 INFO:teuthology.orchestra.run.smithi164.stdout:143 write 0xce86800 thru 0xce87e64 (0x1665 bytes) 2024-09-18T01:02:46.076 INFO:teuthology.orchestra.run.smithi164.stdout:144 write 0x6a04400 thru 0x6a05266 (0xe67 bytes) 2024-09-18T01:02:46.076 INFO:teuthology.orchestra.run.smithi164.stdout:145 punch from 0x17aee00 to 0x17baa00, (0xbc00 bytes) 2024-09-18T01:02:46.076 INFO:teuthology.orchestra.run.smithi164.stdout:146 read 0xa2bea00 thru 0xa2c22d7 (0x38d8 bytes) 2024-09-18T01:02:46.089 INFO:teuthology.orchestra.run.smithi164.stdout:148 trunc from 0xce88000 to 0x1be7200 2024-09-18T01:02:46.107 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:46.107 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:46.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:02:46.137 INFO:teuthology.orchestra.run.smithi164.stdout:149 write 0x7ed2200 thru 0x7edaef1 (0x8cf2 bytes) 2024-09-18T01:02:46.137 INFO:teuthology.orchestra.run.smithi164.stdout:150 read 0x6083e00 thru 0x608f8da (0xbadb bytes) 2024-09-18T01:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout:155 write 0xa6b1800 thru 0xa6bd734 (0xbf35 bytes) 2024-09-18T01:02:46.159 INFO:teuthology.orchestra.run.smithi164.stdout:156 write 0x4f58400 thru 0x4f5b7c3 (0x33c4 bytes) 2024-09-18T01:02:46.176 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:46.175+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:46.182 INFO:teuthology.orchestra.run.smithi164.stdout:158 write 0xdbd4a00 thru 0xdbdd707 (0x8d08 bytes) 2024-09-18T01:02:46.190 INFO:teuthology.orchestra.run.smithi164.stdout:161 trunc from 0xdbdd800 to 0x6aa7400 2024-09-18T01:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout:164 read 0x6936c00 thru 0x693d0e2 (0x64e3 bytes) 2024-09-18T01:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout:165 read 0xd75200 thru 0xd82966 (0xd767 bytes) 2024-09-18T01:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout:166 punch from 0x541c000 to 0x5423a00, (0x7a00 bytes) 2024-09-18T01:02:46.216 INFO:teuthology.orchestra.run.smithi164.stdout:167 read 0x3014200 thru 0x301fa2f (0xb830 bytes) 2024-09-18T01:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout:168 write 0x2d7ba00 thru 0x2d8a63b (0xec3c bytes) 2024-09-18T01:02:46.217 INFO:teuthology.orchestra.run.smithi164.stdout:171 clone 1 order 19 su 131072 sc 14 2024-09-18T01:02:46.225 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:46.226 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:46.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:02:46.277 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:46.277+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:46.341 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:46.342 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:46.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:02:46.378 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:46.377+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:46.457 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:46.458 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:46.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:02:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:02:46.479 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:46.478+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:46.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:02:46.493 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:02:46.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:02:46.579 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:46.578+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:46.581 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:46.581 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:46.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:02:46.680 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:46.680+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:46.746 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:46.746 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:46.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:02:46.796 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:46.795+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:46.837 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:02:46.849 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-18T01:02:46.477513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":181,"num_read_kb":152,"num_write":167,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163704,"kb_used_data":1508,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067336,"statfs":{"total":579820584960,"available":579652952064,"internally_reserved":0,"allocated":1544192,"data_stored":1676222,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023765"},"pg_stats":[{"pgid":"3.1f","version":"23'1","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.653679+0000","last_change":"2024-09-18T01:02:43.469611+0000","last_active":"2024-09-18T01:02:43.653679+0000","last_peered":"2024-09-18T01:02:43.653679+0000","last_clean":"2024-09-18T01:02:43.653679+0000","last_became_active":"2024-09-18T01:02:41.451184+0000","last_became_peered":"2024-09-18T01:02:41.451184+0000","last_unstale":"2024-09-18T01:02:43.653679+0000","last_undegraded":"2024-09-18T01:02:43.653679+0000","last_fullsized":"2024-09-18T01:02:43.653679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:59:42.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005167,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469963+0000","last_change":"2024-09-18T01:02:43.469963+0000","last_active":"2024-09-18T01:02:43.469963+0000","last_peered":"2024-09-18T01:02:43.469963+0000","last_clean":"2024-09-18T01:02:43.469963+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:02:43.469963+0000","last_undegraded":"2024-09-18T01:02:43.469963+0000","last_fullsized":"2024-09-18T01:02:43.469963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:47.725845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013026730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480292+0000","last_change":"2024-09-18T01:02:43.480292+0000","last_active":"2024-09-18T01:02:43.480292+0000","last_peered":"2024-09-18T01:02:43.480292+0000","last_clean":"2024-09-18T01:02:43.480292+0000","last_became_active":"2024-09-18T01:02:41.459434+0000","last_became_peered":"2024-09-18T01:02:41.459434+0000","last_unstale":"2024-09-18T01:02:43.480292+0000","last_undegraded":"2024-09-18T01:02:43.480292+0000","last_fullsized":"2024-09-18T01:02:43.480292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:37.426143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918460000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"23'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.484931+0000","last_change":"2024-09-18T01:02:43.480065+0000","last_active":"2024-09-18T01:02:43.484931+0000","last_peered":"2024-09-18T01:02:43.484931+0000","last_clean":"2024-09-18T01:02:43.484931+0000","last_became_active":"2024-09-18T01:02:41.459497+0000","last_became_peered":"2024-09-18T01:02:41.459497+0000","last_unstale":"2024-09-18T01:02:43.484931+0000","last_undegraded":"2024-09-18T01:02:43.484931+0000","last_fullsized":"2024-09-18T01:02:43.484931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:17.043439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242879999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469715+0000","last_change":"2024-09-18T01:02:43.469715+0000","last_active":"2024-09-18T01:02:43.469715+0000","last_peered":"2024-09-18T01:02:43.469715+0000","last_clean":"2024-09-18T01:02:43.469715+0000","last_became_active":"2024-09-18T01:02:41.454655+0000","last_became_peered":"2024-09-18T01:02:41.454655+0000","last_unstale":"2024-09-18T01:02:43.469715+0000","last_undegraded":"2024-09-18T01:02:43.469715+0000","last_fullsized":"2024-09-18T01:02:43.469715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:42:13.219004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087094899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"23'2","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.701913+0000","last_change":"2024-09-18T01:02:43.474216+0000","last_active":"2024-09-18T01:02:43.701913+0000","last_peered":"2024-09-18T01:02:43.701913+0000","last_clean":"2024-09-18T01:02:43.701913+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:02:43.701913+0000","last_undegraded":"2024-09-18T01:02:43.701913+0000","last_fullsized":"2024-09-18T01:02:43.701913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:33:25.946015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074574000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"23'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.491597+0000","last_change":"2024-09-18T01:02:43.475997+0000","last_active":"2024-09-18T01:02:43.491597+0000","last_peered":"2024-09-18T01:02:43.491597+0000","last_clean":"2024-09-18T01:02:43.491597+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:02:43.491597+0000","last_undegraded":"2024-09-18T01:02:43.491597+0000","last_fullsized":"2024-09-18T01:02:43.491597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:32.555474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038270300000000003,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468721+0000","last_change":"2024-09-18T01:02:43.468721+0000","last_active":"2024-09-18T01:02:43.468721+0000","last_peered":"2024-09-18T01:02:43.468721+0000","last_clean":"2024-09-18T01:02:43.468721+0000","last_became_active":"2024-09-18T01:02:41.459321+0000","last_became_peered":"2024-09-18T01:02:41.459321+0000","last_unstale":"2024-09-18T01:02:43.468721+0000","last_undegraded":"2024-09-18T01:02:43.468721+0000","last_fullsized":"2024-09-18T01:02:43.468721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:34.863151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061623500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469864+0000","last_change":"2024-09-18T01:02:43.469864+0000","last_active":"2024-09-18T01:02:43.469864+0000","last_peered":"2024-09-18T01:02:43.469864+0000","last_clean":"2024-09-18T01:02:43.469864+0000","last_became_active":"2024-09-18T01:02:41.452369+0000","last_became_peered":"2024-09-18T01:02:41.452369+0000","last_unstale":"2024-09-18T01:02:43.469864+0000","last_undegraded":"2024-09-18T01:02:43.469864+0000","last_fullsized":"2024-09-18T01:02:43.469864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:40:40.061157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097728599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480058+0000","last_change":"2024-09-18T01:02:43.480058+0000","last_active":"2024-09-18T01:02:43.480058+0000","last_peered":"2024-09-18T01:02:43.480058+0000","last_clean":"2024-09-18T01:02:43.480058+0000","last_became_active":"2024-09-18T01:02:41.458673+0000","last_became_peered":"2024-09-18T01:02:41.458673+0000","last_unstale":"2024-09-18T01:02:43.480058+0000","last_undegraded":"2024-09-18T01:02:43.480058+0000","last_fullsized":"2024-09-18T01:02:43.480058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:19.330581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099035500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464371+0000","last_change":"2024-09-18T01:02:43.464371+0000","last_active":"2024-09-18T01:02:43.464371+0000","last_peered":"2024-09-18T01:02:43.464371+0000","last_clean":"2024-09-18T01:02:43.464371+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:02:43.464371+0000","last_undegraded":"2024-09-18T01:02:43.464371+0000","last_fullsized":"2024-09-18T01:02:43.464371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:34.144441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077757900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464338+0000","last_change":"2024-09-18T01:02:43.464338+0000","last_active":"2024-09-18T01:02:43.464338+0000","last_peered":"2024-09-18T01:02:43.464338+0000","last_clean":"2024-09-18T01:02:43.464338+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:02:43.464338+0000","last_undegraded":"2024-09-18T01:02:43.464338+0000","last_fullsized":"2024-09-18T01:02:43.464338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:36.483113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080305299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463661+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:02:43.463661+0000","last_peered":"2024-09-18T01:02:43.463661+0000","last_clean":"2024-09-18T01:02:43.463661+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:02:43.463661+0000","last_undegraded":"2024-09-18T01:02:43.463661+0000","last_fullsized":"2024-09-18T01:02:43.463661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469252+0000","last_change":"2024-09-18T01:02:43.469252+0000","last_active":"2024-09-18T01:02:43.469252+0000","last_peered":"2024-09-18T01:02:43.469252+0000","last_clean":"2024-09-18T01:02:43.469252+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:02:43.469252+0000","last_undegraded":"2024-09-18T01:02:43.469252+0000","last_fullsized":"2024-09-18T01:02:43.469252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:11.120138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066086499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476166+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:02:43.476166+0000","last_peered":"2024-09-18T01:02:43.476166+0000","last_clean":"2024-09-18T01:02:43.476166+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:02:43.476166+0000","last_undegraded":"2024-09-18T01:02:43.476166+0000","last_fullsized":"2024-09-18T01:02:43.476166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469995+0000","last_change":"2024-09-18T01:02:43.469995+0000","last_active":"2024-09-18T01:02:43.469995+0000","last_peered":"2024-09-18T01:02:43.469995+0000","last_clean":"2024-09-18T01:02:43.469995+0000","last_became_active":"2024-09-18T01:02:41.456801+0000","last_became_peered":"2024-09-18T01:02:41.456801+0000","last_unstale":"2024-09-18T01:02:43.469995+0000","last_undegraded":"2024-09-18T01:02:43.469995+0000","last_fullsized":"2024-09-18T01:02:43.469995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:00.152087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159869,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463675+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:02:43.463675+0000","last_peered":"2024-09-18T01:02:43.463675+0000","last_clean":"2024-09-18T01:02:43.463675+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:02:43.463675+0000","last_undegraded":"2024-09-18T01:02:43.463675+0000","last_fullsized":"2024-09-18T01:02:43.463675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476334+0000","last_change":"2024-09-18T01:02:43.476334+0000","last_active":"2024-09-18T01:02:43.476334+0000","last_peered":"2024-09-18T01:02:43.476334+0000","last_clean":"2024-09-18T01:02:43.476334+0000","last_became_active":"2024-09-18T01:02:41.457022+0000","last_became_peered":"2024-09-18T01:02:41.457022+0000","last_unstale":"2024-09-18T01:02:43.476334+0000","last_undegraded":"2024-09-18T01:02:43.476334+0000","last_fullsized":"2024-09-18T01:02:43.476334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.197810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050705100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.475801+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:02:43.475801+0000","last_peered":"2024-09-18T01:02:43.475801+0000","last_clean":"2024-09-18T01:02:43.475801+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:02:43.475801+0000","last_undegraded":"2024-09-18T01:02:43.475801+0000","last_fullsized":"2024-09-18T01:02:43.475801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480602+0000","last_change":"2024-09-18T01:02:43.480602+0000","last_active":"2024-09-18T01:02:43.480602+0000","last_peered":"2024-09-18T01:02:43.480602+0000","last_clean":"2024-09-18T01:02:43.480602+0000","last_became_active":"2024-09-18T01:02:41.457580+0000","last_became_peered":"2024-09-18T01:02:41.457580+0000","last_unstale":"2024-09-18T01:02:43.480602+0000","last_undegraded":"2024-09-18T01:02:43.480602+0000","last_fullsized":"2024-09-18T01:02:43.480602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:04:16.185332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013108729999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"23'3","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.704383+0000","last_change":"2024-09-18T01:02:43.473795+0000","last_active":"2024-09-18T01:02:43.704383+0000","last_peered":"2024-09-18T01:02:43.704383+0000","last_clean":"2024-09-18T01:02:43.704383+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:02:43.704383+0000","last_undegraded":"2024-09-18T01:02:43.704383+0000","last_fullsized":"2024-09-18T01:02:43.704383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:18.045070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031347599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.478962+0000","last_change":"2024-09-18T01:02:34.382376+0000","last_active":"2024-09-18T01:02:43.478962+0000","last_peered":"2024-09-18T01:02:43.478962+0000","last_clean":"2024-09-18T01:02:43.478962+0000","last_became_active":"2024-09-18T01:02:32.377565+0000","last_became_peered":"2024-09-18T01:02:32.377565+0000","last_unstale":"2024-09-18T01:02:43.478962+0000","last_undegraded":"2024-09-18T01:02:43.478962+0000","last_fullsized":"2024-09-18T01:02:43.478962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:56.817912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001120694,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476102+0000","last_change":"2024-09-18T01:02:43.476102+0000","last_active":"2024-09-18T01:02:43.476102+0000","last_peered":"2024-09-18T01:02:43.476102+0000","last_clean":"2024-09-18T01:02:43.476102+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:02:43.476102+0000","last_undegraded":"2024-09-18T01:02:43.476102+0000","last_fullsized":"2024-09-18T01:02:43.476102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:44.566585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043811300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468425+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:02:43.468425+0000","last_peered":"2024-09-18T01:02:43.468425+0000","last_clean":"2024-09-18T01:02:43.468425+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:02:43.468425+0000","last_undegraded":"2024-09-18T01:02:43.468425+0000","last_fullsized":"2024-09-18T01:02:43.468425+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468762+0000","last_change":"2024-09-18T01:02:43.468762+0000","last_active":"2024-09-18T01:02:43.468762+0000","last_peered":"2024-09-18T01:02:43.468762+0000","last_clean":"2024-09-18T01:02:43.468762+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:02:43.468762+0000","last_undegraded":"2024-09-18T01:02:43.468762+0000","last_fullsized":"2024-09-18T01:02:43.468762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:10.851384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463270+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:02:43.463270+0000","last_peered":"2024-09-18T01:02:43.463270+0000","last_clean":"2024-09-18T01:02:43.463270+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:02:43.463270+0000","last_undegraded":"2024-09-18T01:02:43.463270+0000","last_fullsized":"2024-09-18T01:02:43.463270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476243+0000","last_change":"2024-09-18T01:02:43.476243+0000","last_active":"2024-09-18T01:02:43.476243+0000","last_peered":"2024-09-18T01:02:43.476243+0000","last_clean":"2024-09-18T01:02:43.476243+0000","last_became_active":"2024-09-18T01:02:41.455199+0000","last_became_peered":"2024-09-18T01:02:41.455199+0000","last_unstale":"2024-09-18T01:02:43.476243+0000","last_undegraded":"2024-09-18T01:02:43.476243+0000","last_fullsized":"2024-09-18T01:02:43.476243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:18.452445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049608300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.479227+0000","last_change":"2024-09-18T01:02:34.381765+0000","last_active":"2024-09-18T01:02:43.479227+0000","last_peered":"2024-09-18T01:02:43.479227+0000","last_clean":"2024-09-18T01:02:43.479227+0000","last_became_active":"2024-09-18T01:02:32.377256+0000","last_became_peered":"2024-09-18T01:02:32.377256+0000","last_unstale":"2024-09-18T01:02:43.479227+0000","last_undegraded":"2024-09-18T01:02:43.479227+0000","last_fullsized":"2024-09-18T01:02:43.479227+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:00.553619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050918999999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463278+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:02:43.463278+0000","last_peered":"2024-09-18T01:02:43.463278+0000","last_clean":"2024-09-18T01:02:43.463278+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:02:43.463278+0000","last_undegraded":"2024-09-18T01:02:43.463278+0000","last_fullsized":"2024-09-18T01:02:43.463278+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468997+0000","last_change":"2024-09-18T01:02:43.468997+0000","last_active":"2024-09-18T01:02:43.468997+0000","last_peered":"2024-09-18T01:02:43.468997+0000","last_clean":"2024-09-18T01:02:43.468997+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:02:43.468997+0000","last_undegraded":"2024-09-18T01:02:43.468997+0000","last_fullsized":"2024-09-18T01:02:43.468997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:40.685553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063056400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.474261+0000","last_change":"2024-09-18T01:02:43.474261+0000","last_active":"2024-09-18T01:02:43.474261+0000","last_peered":"2024-09-18T01:02:43.474261+0000","last_clean":"2024-09-18T01:02:43.474261+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:02:43.474261+0000","last_undegraded":"2024-09-18T01:02:43.474261+0000","last_fullsized":"2024-09-18T01:02:43.474261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:10.599516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065863900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464113+0000","last_change":"2024-09-18T01:02:43.464113+0000","last_active":"2024-09-18T01:02:43.464113+0000","last_peered":"2024-09-18T01:02:43.464113+0000","last_clean":"2024-09-18T01:02:43.464113+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:02:43.464113+0000","last_undegraded":"2024-09-18T01:02:43.464113+0000","last_fullsized":"2024-09-18T01:02:43.464113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:06.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066269099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476485+0000","last_change":"2024-09-18T01:02:43.476485+0000","last_active":"2024-09-18T01:02:43.476485+0000","last_peered":"2024-09-18T01:02:43.476485+0000","last_clean":"2024-09-18T01:02:43.476485+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:02:43.476485+0000","last_undegraded":"2024-09-18T01:02:43.476485+0000","last_fullsized":"2024-09-18T01:02:43.476485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:18.359504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054801899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.479465+0000","last_change":"2024-09-18T01:02:43.479465+0000","last_active":"2024-09-18T01:02:43.479465+0000","last_peered":"2024-09-18T01:02:43.479465+0000","last_clean":"2024-09-18T01:02:43.479465+0000","last_became_active":"2024-09-18T01:02:41.460319+0000","last_became_peered":"2024-09-18T01:02:41.460319+0000","last_unstale":"2024-09-18T01:02:43.479465+0000","last_undegraded":"2024-09-18T01:02:43.479465+0000","last_fullsized":"2024-09-18T01:02:43.479465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:33.510580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043103,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464136+0000","last_change":"2024-09-18T01:02:43.464136+0000","last_active":"2024-09-18T01:02:43.464136+0000","last_peered":"2024-09-18T01:02:43.464136+0000","last_clean":"2024-09-18T01:02:43.464136+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:02:43.464136+0000","last_undegraded":"2024-09-18T01:02:43.464136+0000","last_fullsized":"2024-09-18T01:02:43.464136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:25.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061418400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480515+0000","last_change":"2024-09-18T01:02:43.480515+0000","last_active":"2024-09-18T01:02:43.480515+0000","last_peered":"2024-09-18T01:02:43.480515+0000","last_clean":"2024-09-18T01:02:43.480515+0000","last_became_active":"2024-09-18T01:02:41.458557+0000","last_became_peered":"2024-09-18T01:02:41.458557+0000","last_unstale":"2024-09-18T01:02:43.480515+0000","last_undegraded":"2024-09-18T01:02:43.480515+0000","last_fullsized":"2024-09-18T01:02:43.480515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:46:53.451516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013236470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480377+0000","last_change":"2024-09-18T01:02:43.480377+0000","last_active":"2024-09-18T01:02:43.480377+0000","last_peered":"2024-09-18T01:02:43.480377+0000","last_clean":"2024-09-18T01:02:43.480377+0000","last_became_active":"2024-09-18T01:02:41.458703+0000","last_became_peered":"2024-09-18T01:02:41.458703+0000","last_unstale":"2024-09-18T01:02:43.480377+0000","last_undegraded":"2024-09-18T01:02:43.480377+0000","last_fullsized":"2024-09-18T01:02:43.480377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:26.119620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011817170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.479461+0000","last_change":"2024-09-18T01:02:43.479461+0000","last_active":"2024-09-18T01:02:43.479461+0000","last_peered":"2024-09-18T01:02:43.479461+0000","last_clean":"2024-09-18T01:02:43.479461+0000","last_became_active":"2024-09-18T01:02:41.460416+0000","last_became_peered":"2024-09-18T01:02:41.460416+0000","last_unstale":"2024-09-18T01:02:43.479461+0000","last_undegraded":"2024-09-18T01:02:43.479461+0000","last_fullsized":"2024-09-18T01:02:43.479461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:52.614613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053670300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463798+0000","last_change":"2024-09-18T01:02:43.463798+0000","last_active":"2024-09-18T01:02:43.463798+0000","last_peered":"2024-09-18T01:02:43.463798+0000","last_clean":"2024-09-18T01:02:43.463798+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:02:43.463798+0000","last_undegraded":"2024-09-18T01:02:43.463798+0000","last_fullsized":"2024-09-18T01:02:43.463798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:17:55.244128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042487699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468742+0000","last_change":"2024-09-18T01:02:43.468742+0000","last_active":"2024-09-18T01:02:43.468742+0000","last_peered":"2024-09-18T01:02:43.468742+0000","last_clean":"2024-09-18T01:02:43.468742+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:02:43.468742+0000","last_undegraded":"2024-09-18T01:02:43.468742+0000","last_fullsized":"2024-09-18T01:02:43.468742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.458184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043475499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463865+0000","last_change":"2024-09-18T01:02:43.463865+0000","last_active":"2024-09-18T01:02:43.463865+0000","last_peered":"2024-09-18T01:02:43.463865+0000","last_clean":"2024-09-18T01:02:43.463865+0000","last_became_active":"2024-09-18T01:02:41.452844+0000","last_became_peered":"2024-09-18T01:02:41.452844+0000","last_unstale":"2024-09-18T01:02:43.463865+0000","last_undegraded":"2024-09-18T01:02:43.463865+0000","last_fullsized":"2024-09-18T01:02:43.463865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:25:09.401459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640262,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":82577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640264,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640263,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":672942,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":82558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640264,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640263,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":672942,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":49,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":49,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:02:46.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:02:46.890 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:46.889+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:46.903 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:46.903 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:46.913 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:02:46.979 INFO:teuthology.orchestra.run.smithi164.stdout:172 write 0x9328c00 thru 0x932c1b8 (0x35b9 bytes) 2024-09-18T01:02:46.981 INFO:teuthology.orchestra.run.smithi164.stdout:174 punch from 0x74a4000 to 0x74a7800, (0x3800 bytes) 2024-09-18T01:02:46.983 INFO:teuthology.orchestra.run.smithi164.stdout:180 trunc from 0x932c200 to 0xa9de000 2024-09-18T01:02:46.983 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:46.982+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:46.996 INFO:teuthology.orchestra.run.smithi164.stdout:181 read 0x89e6200 thru 0x89e673f (0x540 bytes) 2024-09-18T01:02:47.001 INFO:teuthology.orchestra.run.smithi164.stdout:183 read 0x5139e00 thru 0x513beb0 (0x20b1 bytes) 2024-09-18T01:02:47.002 INFO:teuthology.orchestra.run.smithi164.stdout:185 write 0x8ac3000 thru 0x8acd96c (0xa96d bytes) 2024-09-18T01:02:47.002 INFO:teuthology.orchestra.run.smithi164.stdout:187 read 0x5230600 thru 0x523097a (0x37b bytes) 2024-09-18T01:02:47.002 INFO:teuthology.orchestra.run.smithi164.stdout:189 read 0x7b62a00 thru 0x7b63b65 (0x1166 bytes) 2024-09-18T01:02:47.002 INFO:teuthology.orchestra.run.smithi164.stdout:191 write 0x3df4400 thru 0x3df4cb6 (0x8b7 bytes) 2024-09-18T01:02:47.005 INFO:teuthology.orchestra.run.smithi164.stdout:193 trunc from 0xa9de000 to 0xce80200 2024-09-18T01:02:47.021 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:47.021 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:47.021 INFO:teuthology.orchestra.run.smithi164.stdout:195 trunc from 0xce80200 to 0xc619e00 2024-09-18T01:02:47.030 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:02:47.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:02:47.036 INFO:teuthology.orchestra.run.smithi164.stdout:196 read 0x321f000 thru 0x322bb90 (0xcb91 bytes) 2024-09-18T01:02:47.041 INFO:teuthology.orchestra.run.smithi164.stdout:199 punch from 0x5844e00 to 0x5854200, (0xf400 bytes) 2024-09-18T01:02:47.041 INFO:teuthology.orchestra.run.smithi164.stdout:201 write 0x54c1c00 thru 0x54cc0de (0xa4df bytes) 2024-09-18T01:02:47.042 INFO:teuthology.orchestra.run.smithi164.stdout:203 punch from 0x2361e00 to 0x2368000, (0x6200 bytes) 2024-09-18T01:02:47.055 INFO:teuthology.orchestra.run.smithi164.stdout:205 trunc from 0xc619e00 to 0x15f0400 2024-09-18T01:02:47.084 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:47.083+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:47.091 INFO:teuthology.orchestra.run.smithi164.stdout:210 trunc from 0x15f0400 to 0x1faa800 2024-09-18T01:02:47.105 INFO:teuthology.orchestra.run.smithi164.stdout:211 punch from 0xda7400 to 0xdb4200, (0xce00 bytes) 2024-09-18T01:02:47.111 INFO:teuthology.orchestra.run.smithi164.stdout:212 punch from 0x11be600 to 0x11c6000, (0x7a00 bytes) 2024-09-18T01:02:47.111 INFO:teuthology.orchestra.run.smithi164.stdout:214 read 0x767400 thru 0x76c168 (0x4d69 bytes) 2024-09-18T01:02:47.111 INFO:teuthology.orchestra.run.smithi164.stdout:216 read 0x184f200 thru 0x1852a25 (0x3826 bytes) 2024-09-18T01:02:47.131 INFO:teuthology.orchestra.run.smithi164.stdout:220 write 0xb1e1e00 thru 0xb1e22f1 (0x4f2 bytes) 2024-09-18T01:02:47.137 INFO:teuthology.orchestra.run.smithi164.stdout:222 read 0x6ac6e00 thru 0x6acd00a (0x620b bytes) 2024-09-18T01:02:47.137 INFO:teuthology.orchestra.run.smithi164.stdout:223 punch from 0x5271800 to 0x527be00, (0xa600 bytes) 2024-09-18T01:02:47.147 INFO:teuthology.orchestra.run.smithi164.stdout:225 trunc from 0xb1e2400 to 0x2f72a00 2024-09-18T01:02:47.164 INFO:teuthology.orchestra.run.smithi164.stdout:226 punch from 0x1c52000 to 0x1c58600, (0x6600 bytes) 2024-09-18T01:02:47.169 INFO:teuthology.orchestra.run.smithi164.stdout:227 read 0x17d5200 thru 0x17d6a63 (0x1864 bytes) 2024-09-18T01:02:47.179 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:47.179 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:47.180 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:47.180 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:47.182 INFO:teuthology.orchestra.run.smithi164.stdout:228 write 0x5545200 thru 0x5553cb4 (0xeab5 bytes) 2024-09-18T01:02:47.185 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:47.184+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:47.189 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:02:47.192 INFO:teuthology.orchestra.run.smithi164.stdout:231 trunc from 0x5553e00 to 0x98da200 2024-09-18T01:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout:238 write 0x8c6a000 thru 0x8c704d4 (0x64d5 bytes) 2024-09-18T01:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout:241 write 0x569d000 thru 0x569d807 (0x808 bytes) 2024-09-18T01:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout:242 punch from 0x186ee00 to 0x1870c00, (0x1e00 bytes) 2024-09-18T01:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout:245 write 0x79e0200 thru 0x79e726c (0x706d bytes) 2024-09-18T01:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout:248 read 0x633b600 thru 0x6344fad (0x99ae bytes) 2024-09-18T01:02:47.211 INFO:teuthology.orchestra.run.smithi164.stdout:249 read 0x95af800 thru 0x95b5804 (0x6005 bytes) 2024-09-18T01:02:47.212 INFO:teuthology.orchestra.run.smithi164.stdout:250 trunc from 0x98da200 to 0x9e28400 2024-09-18T01:02:47.216 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:47.216 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:47.235 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout:251 read 0x6018200 thru 0x60231c5 (0xafc6 bytes) 2024-09-18T01:02:47.236 INFO:teuthology.orchestra.run.smithi164.stdout:253 write 0x8a86200 thru 0x8a9442e (0xe22f bytes) 2024-09-18T01:02:47.241 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:47.244 INFO:teuthology.orchestra.run.smithi164.stdout:254 trunc from 0x9e28400 to 0xf78400 2024-09-18T01:02:47.248 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-18T01:02:46.477513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":181,"num_read_kb":152,"num_write":167,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163704,"kb_used_data":1508,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067336,"statfs":{"total":579820584960,"available":579652952064,"internally_reserved":0,"allocated":1544192,"data_stored":1676222,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023765"},"pg_stats":[{"pgid":"3.1f","version":"23'1","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.653679+0000","last_change":"2024-09-18T01:02:43.469611+0000","last_active":"2024-09-18T01:02:43.653679+0000","last_peered":"2024-09-18T01:02:43.653679+0000","last_clean":"2024-09-18T01:02:43.653679+0000","last_became_active":"2024-09-18T01:02:41.451184+0000","last_became_peered":"2024-09-18T01:02:41.451184+0000","last_unstale":"2024-09-18T01:02:43.653679+0000","last_undegraded":"2024-09-18T01:02:43.653679+0000","last_fullsized":"2024-09-18T01:02:43.653679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:59:42.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005167,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469963+0000","last_change":"2024-09-18T01:02:43.469963+0000","last_active":"2024-09-18T01:02:43.469963+0000","last_peered":"2024-09-18T01:02:43.469963+0000","last_clean":"2024-09-18T01:02:43.469963+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:02:43.469963+0000","last_undegraded":"2024-09-18T01:02:43.469963+0000","last_fullsized":"2024-09-18T01:02:43.469963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:47.725845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013026730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480292+0000","last_change":"2024-09-18T01:02:43.480292+0000","last_active":"2024-09-18T01:02:43.480292+0000","last_peered":"2024-09-18T01:02:43.480292+0000","last_clean":"2024-09-18T01:02:43.480292+0000","last_became_active":"2024-09-18T01:02:41.459434+0000","last_became_peered":"2024-09-18T01:02:41.459434+0000","last_unstale":"2024-09-18T01:02:43.480292+0000","last_undegraded":"2024-09-18T01:02:43.480292+0000","last_fullsized":"2024-09-18T01:02:43.480292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:37.426143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918460000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"23'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.484931+0000","last_change":"2024-09-18T01:02:43.480065+0000","last_active":"2024-09-18T01:02:43.484931+0000","last_peered":"2024-09-18T01:02:43.484931+0000","last_clean":"2024-09-18T01:02:43.484931+0000","last_became_active":"2024-09-18T01:02:41.459497+0000","last_became_peered":"2024-09-18T01:02:41.459497+0000","last_unstale":"2024-09-18T01:02:43.484931+0000","last_undegraded":"2024-09-18T01:02:43.484931+0000","last_fullsized":"2024-09-18T01:02:43.484931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:17.043439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242879999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469715+0000","last_change":"2024-09-18T01:02:43.469715+0000","last_active":"2024-09-18T01:02:43.469715+0000","last_peered":"2024-09-18T01:02:43.469715+0000","last_clean":"2024-09-18T01:02:43.469715+0000","last_became_active":"2024-09-18T01:02:41.454655+0000","last_became_peered":"2024-09-18T01:02:41.454655+0000","last_unstale":"2024-09-18T01:02:43.469715+0000","last_undegraded":"2024-09-18T01:02:43.469715+0000","last_fullsized":"2024-09-18T01:02:43.469715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:42:13.219004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087094899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"23'2","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.701913+0000","last_change":"2024-09-18T01:02:43.474216+0000","last_active":"2024-09-18T01:02:43.701913+0000","last_peered":"2024-09-18T01:02:43.701913+0000","last_clean":"2024-09-18T01:02:43.701913+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:02:43.701913+0000","last_undegraded":"2024-09-18T01:02:43.701913+0000","last_fullsized":"2024-09-18T01:02:43.701913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:33:25.946015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074574000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"23'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.491597+0000","last_change":"2024-09-18T01:02:43.475997+0000","last_active":"2024-09-18T01:02:43.491597+0000","last_peered":"2024-09-18T01:02:43.491597+0000","last_clean":"2024-09-18T01:02:43.491597+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:02:43.491597+0000","last_undegraded":"2024-09-18T01:02:43.491597+0000","last_fullsized":"2024-09-18T01:02:43.491597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:32.555474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038270300000000003,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468721+0000","last_change":"2024-09-18T01:02:43.468721+0000","last_active":"2024-09-18T01:02:43.468721+0000","last_peered":"2024-09-18T01:02:43.468721+0000","last_clean":"2024-09-18T01:02:43.468721+0000","last_became_active":"2024-09-18T01:02:41.459321+0000","last_became_peered":"2024-09-18T01:02:41.459321+0000","last_unstale":"2024-09-18T01:02:43.468721+0000","last_undegraded":"2024-09-18T01:02:43.468721+0000","last_fullsized":"2024-09-18T01:02:43.468721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:34.863151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061623500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469864+0000","last_change":"2024-09-18T01:02:43.469864+0000","last_active":"2024-09-18T01:02:43.469864+0000","last_peered":"2024-09-18T01:02:43.469864+0000","last_clean":"2024-09-18T01:02:43.469864+0000","last_became_active":"2024-09-18T01:02:41.452369+0000","last_became_peered":"2024-09-18T01:02:41.452369+0000","last_unstale":"2024-09-18T01:02:43.469864+0000","last_undegraded":"2024-09-18T01:02:43.469864+0000","last_fullsized":"2024-09-18T01:02:43.469864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:40:40.061157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097728599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480058+0000","last_change":"2024-09-18T01:02:43.480058+0000","last_active":"2024-09-18T01:02:43.480058+0000","last_peered":"2024-09-18T01:02:43.480058+0000","last_clean":"2024-09-18T01:02:43.480058+0000","last_became_active":"2024-09-18T01:02:41.458673+0000","last_became_peered":"2024-09-18T01:02:41.458673+0000","last_unstale":"2024-09-18T01:02:43.480058+0000","last_undegraded":"2024-09-18T01:02:43.480058+0000","last_fullsized":"2024-09-18T01:02:43.480058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:19.330581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099035500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464371+0000","last_change":"2024-09-18T01:02:43.464371+0000","last_active":"2024-09-18T01:02:43.464371+0000","last_peered":"2024-09-18T01:02:43.464371+0000","last_clean":"2024-09-18T01:02:43.464371+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:02:43.464371+0000","last_undegraded":"2024-09-18T01:02:43.464371+0000","last_fullsized":"2024-09-18T01:02:43.464371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:34.144441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077757900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464338+0000","last_change":"2024-09-18T01:02:43.464338+0000","last_active":"2024-09-18T01:02:43.464338+0000","last_peered":"2024-09-18T01:02:43.464338+0000","last_clean":"2024-09-18T01:02:43.464338+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:02:43.464338+0000","last_undegraded":"2024-09-18T01:02:43.464338+0000","last_fullsized":"2024-09-18T01:02:43.464338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:36.483113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080305299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463661+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:02:43.463661+0000","last_peered":"2024-09-18T01:02:43.463661+0000","last_clean":"2024-09-18T01:02:43.463661+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:02:43.463661+0000","last_undegraded":"2024-09-18T01:02:43.463661+0000","last_fullsized":"2024-09-18T01:02:43.463661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469252+0000","last_change":"2024-09-18T01:02:43.469252+0000","last_active":"2024-09-18T01:02:43.469252+0000","last_peered":"2024-09-18T01:02:43.469252+0000","last_clean":"2024-09-18T01:02:43.469252+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:02:43.469252+0000","last_undegraded":"2024-09-18T01:02:43.469252+0000","last_fullsized":"2024-09-18T01:02:43.469252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:11.120138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066086499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476166+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:02:43.476166+0000","last_peered":"2024-09-18T01:02:43.476166+0000","last_clean":"2024-09-18T01:02:43.476166+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:02:43.476166+0000","last_undegraded":"2024-09-18T01:02:43.476166+0000","last_fullsized":"2024-09-18T01:02:43.476166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.469995+0000","last_change":"2024-09-18T01:02:43.469995+0000","last_active":"2024-09-18T01:02:43.469995+0000","last_peered":"2024-09-18T01:02:43.469995+0000","last_clean":"2024-09-18T01:02:43.469995+0000","last_became_active":"2024-09-18T01:02:41.456801+0000","last_became_peered":"2024-09-18T01:02:41.456801+0000","last_unstale":"2024-09-18T01:02:43.469995+0000","last_undegraded":"2024-09-18T01:02:43.469995+0000","last_fullsized":"2024-09-18T01:02:43.469995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:00.152087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159869,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463675+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:02:43.463675+0000","last_peered":"2024-09-18T01:02:43.463675+0000","last_clean":"2024-09-18T01:02:43.463675+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:02:43.463675+0000","last_undegraded":"2024-09-18T01:02:43.463675+0000","last_fullsized":"2024-09-18T01:02:43.463675+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476334+0000","last_change":"2024-09-18T01:02:43.476334+0000","last_active":"2024-09-18T01:02:43.476334+0000","last_peered":"2024-09-18T01:02:43.476334+0000","last_clean":"2024-09-18T01:02:43.476334+0000","last_became_active":"2024-09-18T01:02:41.457022+0000","last_became_peered":"2024-09-18T01:02:41.457022+0000","last_unstale":"2024-09-18T01:02:43.476334+0000","last_undegraded":"2024-09-18T01:02:43.476334+0000","last_fullsized":"2024-09-18T01:02:43.476334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.197810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050705100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.475801+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:02:43.475801+0000","last_peered":"2024-09-18T01:02:43.475801+0000","last_clean":"2024-09-18T01:02:43.475801+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:02:43.475801+0000","last_undegraded":"2024-09-18T01:02:43.475801+0000","last_fullsized":"2024-09-18T01:02:43.475801+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480602+0000","last_change":"2024-09-18T01:02:43.480602+0000","last_active":"2024-09-18T01:02:43.480602+0000","last_peered":"2024-09-18T01:02:43.480602+0000","last_clean":"2024-09-18T01:02:43.480602+0000","last_became_active":"2024-09-18T01:02:41.457580+0000","last_became_peered":"2024-09-18T01:02:41.457580+0000","last_unstale":"2024-09-18T01:02:43.480602+0000","last_undegraded":"2024-09-18T01:02:43.480602+0000","last_fullsized":"2024-09-18T01:02:43.480602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:04:16.185332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013108729999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"23'3","reported_seq":39,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.704383+0000","last_change":"2024-09-18T01:02:43.473795+0000","last_active":"2024-09-18T01:02:43.704383+0000","last_peered":"2024-09-18T01:02:43.704383+0000","last_clean":"2024-09-18T01:02:43.704383+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:02:43.704383+0000","last_undegraded":"2024-09-18T01:02:43.704383+0000","last_fullsized":"2024-09-18T01:02:43.704383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:18.045070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031347599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.478962+0000","last_change":"2024-09-18T01:02:34.382376+0000","last_active":"2024-09-18T01:02:43.478962+0000","last_peered":"2024-09-18T01:02:43.478962+0000","last_clean":"2024-09-18T01:02:43.478962+0000","last_became_active":"2024-09-18T01:02:32.377565+0000","last_became_peered":"2024-09-18T01:02:32.377565+0000","last_unstale":"2024-09-18T01:02:43.478962+0000","last_undegraded":"2024-09-18T01:02:43.478962+0000","last_fullsized":"2024-09-18T01:02:43.478962+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:56.817912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001120694,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476102+0000","last_change":"2024-09-18T01:02:43.476102+0000","last_active":"2024-09-18T01:02:43.476102+0000","last_peered":"2024-09-18T01:02:43.476102+0000","last_clean":"2024-09-18T01:02:43.476102+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:02:43.476102+0000","last_undegraded":"2024-09-18T01:02:43.476102+0000","last_fullsized":"2024-09-18T01:02:43.476102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:44.566585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043811300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468425+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:02:43.468425+0000","last_peered":"2024-09-18T01:02:43.468425+0000","last_clean":"2024-09-18T01:02:43.468425+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:02:43.468425+0000","last_undegraded":"2024-09-18T01:02:43.468425+0000","last_fullsized":"2024-09-18T01:02:43.468425+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468762+0000","last_change":"2024-09-18T01:02:43.468762+0000","last_active":"2024-09-18T01:02:43.468762+0000","last_peered":"2024-09-18T01:02:43.468762+0000","last_clean":"2024-09-18T01:02:43.468762+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:02:43.468762+0000","last_undegraded":"2024-09-18T01:02:43.468762+0000","last_fullsized":"2024-09-18T01:02:43.468762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:10.851384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463270+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:02:43.463270+0000","last_peered":"2024-09-18T01:02:43.463270+0000","last_clean":"2024-09-18T01:02:43.463270+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:02:43.463270+0000","last_undegraded":"2024-09-18T01:02:43.463270+0000","last_fullsized":"2024-09-18T01:02:43.463270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476243+0000","last_change":"2024-09-18T01:02:43.476243+0000","last_active":"2024-09-18T01:02:43.476243+0000","last_peered":"2024-09-18T01:02:43.476243+0000","last_clean":"2024-09-18T01:02:43.476243+0000","last_became_active":"2024-09-18T01:02:41.455199+0000","last_became_peered":"2024-09-18T01:02:41.455199+0000","last_unstale":"2024-09-18T01:02:43.476243+0000","last_undegraded":"2024-09-18T01:02:43.476243+0000","last_fullsized":"2024-09-18T01:02:43.476243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:18.452445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049608300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.479227+0000","last_change":"2024-09-18T01:02:34.381765+0000","last_active":"2024-09-18T01:02:43.479227+0000","last_peered":"2024-09-18T01:02:43.479227+0000","last_clean":"2024-09-18T01:02:43.479227+0000","last_became_active":"2024-09-18T01:02:32.377256+0000","last_became_peered":"2024-09-18T01:02:32.377256+0000","last_unstale":"2024-09-18T01:02:43.479227+0000","last_undegraded":"2024-09-18T01:02:43.479227+0000","last_fullsized":"2024-09-18T01:02:43.479227+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:52:00.553619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050918999999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463278+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:02:43.463278+0000","last_peered":"2024-09-18T01:02:43.463278+0000","last_clean":"2024-09-18T01:02:43.463278+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:02:43.463278+0000","last_undegraded":"2024-09-18T01:02:43.463278+0000","last_fullsized":"2024-09-18T01:02:43.463278+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468997+0000","last_change":"2024-09-18T01:02:43.468997+0000","last_active":"2024-09-18T01:02:43.468997+0000","last_peered":"2024-09-18T01:02:43.468997+0000","last_clean":"2024-09-18T01:02:43.468997+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:02:43.468997+0000","last_undegraded":"2024-09-18T01:02:43.468997+0000","last_fullsized":"2024-09-18T01:02:43.468997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:40.685553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063056400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.474261+0000","last_change":"2024-09-18T01:02:43.474261+0000","last_active":"2024-09-18T01:02:43.474261+0000","last_peered":"2024-09-18T01:02:43.474261+0000","last_clean":"2024-09-18T01:02:43.474261+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:02:43.474261+0000","last_undegraded":"2024-09-18T01:02:43.474261+0000","last_fullsized":"2024-09-18T01:02:43.474261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:10.599516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065863900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464113+0000","last_change":"2024-09-18T01:02:43.464113+0000","last_active":"2024-09-18T01:02:43.464113+0000","last_peered":"2024-09-18T01:02:43.464113+0000","last_clean":"2024-09-18T01:02:43.464113+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:02:43.464113+0000","last_undegraded":"2024-09-18T01:02:43.464113+0000","last_fullsized":"2024-09-18T01:02:43.464113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:06.677823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066269099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.476485+0000","last_change":"2024-09-18T01:02:43.476485+0000","last_active":"2024-09-18T01:02:43.476485+0000","last_peered":"2024-09-18T01:02:43.476485+0000","last_clean":"2024-09-18T01:02:43.476485+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:02:43.476485+0000","last_undegraded":"2024-09-18T01:02:43.476485+0000","last_fullsized":"2024-09-18T01:02:43.476485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:18.359504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054801899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.479465+0000","last_change":"2024-09-18T01:02:43.479465+0000","last_active":"2024-09-18T01:02:43.479465+0000","last_peered":"2024-09-18T01:02:43.479465+0000","last_clean":"2024-09-18T01:02:43.479465+0000","last_became_active":"2024-09-18T01:02:41.460319+0000","last_became_peered":"2024-09-18T01:02:41.460319+0000","last_unstale":"2024-09-18T01:02:43.479465+0000","last_undegraded":"2024-09-18T01:02:43.479465+0000","last_fullsized":"2024-09-18T01:02:43.479465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:33.510580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043103,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.464136+0000","last_change":"2024-09-18T01:02:43.464136+0000","last_active":"2024-09-18T01:02:43.464136+0000","last_peered":"2024-09-18T01:02:43.464136+0000","last_clean":"2024-09-18T01:02:43.464136+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:02:43.464136+0000","last_undegraded":"2024-09-18T01:02:43.464136+0000","last_fullsized":"2024-09-18T01:02:43.464136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:25.152655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061418400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480515+0000","last_change":"2024-09-18T01:02:43.480515+0000","last_active":"2024-09-18T01:02:43.480515+0000","last_peered":"2024-09-18T01:02:43.480515+0000","last_clean":"2024-09-18T01:02:43.480515+0000","last_became_active":"2024-09-18T01:02:41.458557+0000","last_became_peered":"2024-09-18T01:02:41.458557+0000","last_unstale":"2024-09-18T01:02:43.480515+0000","last_undegraded":"2024-09-18T01:02:43.480515+0000","last_fullsized":"2024-09-18T01:02:43.480515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:46:53.451516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013236470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.480377+0000","last_change":"2024-09-18T01:02:43.480377+0000","last_active":"2024-09-18T01:02:43.480377+0000","last_peered":"2024-09-18T01:02:43.480377+0000","last_clean":"2024-09-18T01:02:43.480377+0000","last_became_active":"2024-09-18T01:02:41.458703+0000","last_became_peered":"2024-09-18T01:02:41.458703+0000","last_unstale":"2024-09-18T01:02:43.480377+0000","last_undegraded":"2024-09-18T01:02:43.480377+0000","last_fullsized":"2024-09-18T01:02:43.480377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:26.119620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011817170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.479461+0000","last_change":"2024-09-18T01:02:43.479461+0000","last_active":"2024-09-18T01:02:43.479461+0000","last_peered":"2024-09-18T01:02:43.479461+0000","last_clean":"2024-09-18T01:02:43.479461+0000","last_became_active":"2024-09-18T01:02:41.460416+0000","last_became_peered":"2024-09-18T01:02:41.460416+0000","last_unstale":"2024-09-18T01:02:43.479461+0000","last_undegraded":"2024-09-18T01:02:43.479461+0000","last_fullsized":"2024-09-18T01:02:43.479461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:52.614613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053670300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463798+0000","last_change":"2024-09-18T01:02:43.463798+0000","last_active":"2024-09-18T01:02:43.463798+0000","last_peered":"2024-09-18T01:02:43.463798+0000","last_clean":"2024-09-18T01:02:43.463798+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:02:43.463798+0000","last_undegraded":"2024-09-18T01:02:43.463798+0000","last_fullsized":"2024-09-18T01:02:43.463798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:17:55.244128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042487699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.468742+0000","last_change":"2024-09-18T01:02:43.468742+0000","last_active":"2024-09-18T01:02:43.468742+0000","last_peered":"2024-09-18T01:02:43.468742+0000","last_clean":"2024-09-18T01:02:43.468742+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:02:43.468742+0000","last_undegraded":"2024-09-18T01:02:43.468742+0000","last_fullsized":"2024-09-18T01:02:43.468742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.458184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043475499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-18T01:02:43.463865+0000","last_change":"2024-09-18T01:02:43.463865+0000","last_active":"2024-09-18T01:02:43.463865+0000","last_peered":"2024-09-18T01:02:43.463865+0000","last_clean":"2024-09-18T01:02:43.463865+0000","last_became_active":"2024-09-18T01:02:41.452844+0000","last_became_peered":"2024-09-18T01:02:41.452844+0000","last_unstale":"2024-09-18T01:02:43.463865+0000","last_undegraded":"2024-09-18T01:02:43.463865+0000","last_fullsized":"2024-09-18T01:02:43.463865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:25:09.401459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":51,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640262,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":82577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640264,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640263,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":672942,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":82558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640264,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640263,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":672942,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":49,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":49,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:02:47.250 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:02:47.250 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:02:47.251 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-18T01:02:47.251 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-18T01:02:47.265 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:47.266 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:47.285 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:47.284+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:47.287 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:47.288 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:47.301 INFO:teuthology.orchestra.run.smithi164.stdout:255 write 0x796e400 thru 0x7971655 (0x3256 bytes) 2024-09-18T01:02:47.307 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:47.308 INFO:teuthology.orchestra.run.smithi164.stdout:260 read 0x6172c00 thru 0x617ea1b (0xbe1c bytes) 2024-09-18T01:02:47.308 INFO:teuthology.orchestra.run.smithi164.stdout:261 write 0x50a4a00 thru 0x50b02c3 (0xb8c4 bytes) 2024-09-18T01:02:47.323 INFO:teuthology.orchestra.run.smithi164.stdout:262 write 0x90fe800 thru 0x910a111 (0xb912 bytes) 2024-09-18T01:02:47.330 INFO:teuthology.orchestra.run.smithi164.stdout:263 punch from 0x76d9800 to 0x76dd600, (0x3e00 bytes) 2024-09-18T01:02:47.331 INFO:teuthology.orchestra.run.smithi164.stdout:265 punch from 0x16c4600 to 0x16d4400, (0xfe00 bytes) 2024-09-18T01:02:47.331 INFO:teuthology.orchestra.run.smithi164.stdout:266 read 0x72a0a00 thru 0x72a6632 (0x5c33 bytes) 2024-09-18T01:02:47.334 INFO:teuthology.orchestra.run.smithi164.stdout:267 trunc from 0x910a200 to 0xbdb5600 2024-09-18T01:02:47.342 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:47.342 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:47.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 21.002289042000001, 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0076954079999999999, 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 37 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337902+0000", 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6599999999999999e-06 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337905+0000", 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.371e-06 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337913+0000", 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1580000000000005e-06 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341291+0000", 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033772979999999999 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341344+0000", 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3483999999999998e-05 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341426+0000", 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1462e-05 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.343020+0000", 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015945740000000001 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345556+0000", 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025358749999999999 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345578+0000", 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1865000000000002e-05 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345594+0000", 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6430999999999999e-05 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.808281461, 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077811820000000002, 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.349 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 111 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531911+0000", 2024-09-18T01:02:47.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4929999999999998e-06 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531917+0000", 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5509999999999996e-06 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531934+0000", 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6356000000000001e-05 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535192+0000", 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032581369999999999 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535241+0000", 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9363999999999997e-05 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535318+0000", 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7078999999999998e-05 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538191+0000", 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0028732240000000002 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539619+0000", 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014276619999999999 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539661+0000", 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1819000000000001e-05 2024-09-18T01:02:47.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539688+0000", 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7030999999999999e-05 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.808239296, 2024-09-18T01:02:47.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077932879999999998, 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 112 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:47.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6100000000000001e-07 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532030+0000", 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1082999999999996e-05 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532040+0000", 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9029999999999999e-06 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532047+0000", 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:02:47.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535639+0000", 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0035927009999999998 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535685+0000", 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5661000000000002e-05 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535757+0000", 2024-09-18T01:02:47.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2113999999999994e-05 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538221+0000", 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024644440000000001 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539694+0000", 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001472283 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539719+0000", 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4777999999999999e-05 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539742+0000", 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3547e-05 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.808128263, 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077280339999999999, 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 113 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532061+0000", 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1900000000000002e-07 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532064+0000", 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8720000000000004e-06 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532078+0000", 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3766e-05 2024-09-18T01:02:47.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532085+0000", 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.939e-06 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536066+0000", 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039811229999999996 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536113+0000", 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6292000000000003e-05 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536186+0000", 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3632000000000002e-05 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538252+0000", 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020659419999999999 2024-09-18T01:02:47.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539746+0000", 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014934670000000001 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539770+0000", 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4311000000000002e-05 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539788+0000", 2024-09-18T01:02:47.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7971000000000001e-05 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.808091867000002, 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.007737715, 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.368 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 114 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532097+0000", 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3499999999999996e-07 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532102+0000", 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8770000000000001e-06 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532108+0000", 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.117e-06 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532118+0000", 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0101999999999999e-05 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536401+0000", 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042836200000000001 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536448+0000", 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.638e-05 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536522+0000", 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3769000000000005e-05 2024-09-18T01:02:47.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538275+0000", 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017533819999999999 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539792+0000", 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001516861 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539814+0000", 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2305000000000001e-05 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539834+0000", 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9766999999999999e-05 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.374 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.808050029, 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010209316, 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.375 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 115 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532186+0000", 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7521999999999997e-05 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532192+0000", 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5049999999999998e-06 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532195+0000", 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5159999999999999e-06 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536736+0000", 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0045408999999999996 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536783+0000", 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6956999999999998e-05 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536851+0000", 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538944+0000", 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020932110000000002 2024-09-18T01:02:47.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542279+0000", 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033348679999999999 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542321+0000", 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1520999999999998e-05 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542347+0000", 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6789000000000001e-05 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.807986035999999, 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010197308, 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.381 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 116 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532203+0000", 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532205+0000", 2024-09-18T01:02:47.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5610000000000001e-06 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532209+0000", 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9639999999999997e-06 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532213+0000", 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5939999999999998e-06 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537234+0000", 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050209440000000003 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537281+0000", 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6955e-05 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537352+0000", 2024-09-18T01:02:47.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1870000000000001e-05 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538964+0000", 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001611517 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542353+0000", 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003389076 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542376+0000", 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3456e-05 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542399+0000", 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2935e-05 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.387 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.807970934, 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010229594999999999, 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 117 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.03e-07 2024-09-18T01:02:47.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532220+0000", 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1710000000000001e-06 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532223+0000", 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4740000000000001e-06 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532227+0000", 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5920000000000001e-06 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537567+0000", 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053407589999999996 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537615+0000", 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7590999999999999e-05 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537694+0000", 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9015000000000002e-05 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538995+0000", 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013009429999999999 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542403+0000", 2024-09-18T01:02:47.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034077180000000001 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542428+0000", 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5607000000000001e-05 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542447+0000", 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8422000000000001e-05 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.393 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.336240457999999, 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0095908409999999993, 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.394 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 130 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0500000000000002e-07 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003953+0000", 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2860000000000001e-06 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003964+0000", 2024-09-18T01:02:47.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1259e-05 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003970+0000", 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1349999999999997e-06 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004461+0000", 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00049077799999999998 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004507+0000", 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5677000000000003e-05 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004590+0000", 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3610999999999996e-05 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006628+0000", 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020381240000000001 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013439+0000", 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.006810633 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013498+0000", 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9101999999999998e-05 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013538+0000", 2024-09-18T01:02:47.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0431e-05 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:47.400 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.336202211, 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096617260000000007, 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 131 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:47.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003987+0000", 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004027+0000", 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0516999999999998e-05 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004034+0000", 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5220000000000004e-06 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004038+0000", 2024-09-18T01:02:47.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0960000000000003e-06 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004894+0000", 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00085607399999999996 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004982+0000", 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7811000000000003e-05 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005115+0000", 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013318600000000001 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006634+0000", 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001519643 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013579+0000", 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0069450140000000002 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013610+0000", 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0849999999999998e-05 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013648+0000", 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7372000000000001e-05 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.336143454999998, 2024-09-18T01:02:47.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096891579999999998, 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 132 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.15e-07 2024-09-18T01:02:47.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004047+0000", 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.232e-06 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004051+0000", 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.58e-06 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004056+0000", 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0839999999999999e-06 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005528+0000", 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014725350000000001 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005573+0000", 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4518999999999998e-05 2024-09-18T01:02:47.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005648+0000", 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4757000000000003e-05 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007604+0000", 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019566739999999998 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013684+0000", 2024-09-18T01:02:47.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0060797519999999999 2024-09-18T01:02:47.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013712+0000", 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8214000000000001e-05 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013734+0000", 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1495999999999999e-05 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.336020503, 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096596410000000001, 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.417 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 133 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:47.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4099999999999998e-07 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004173+0000", 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4270000000000001e-06 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004188+0000", 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5435999999999999e-05 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004198+0000", 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0034000000000001e-05 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006144+0000", 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019458489999999999 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006225+0000", 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1207999999999999e-05 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006346+0000", 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012035400000000001 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008493+0000", 2024-09-18T01:02:47.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021473849999999999 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013769+0000", 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052760510000000004 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013795+0000", 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6205000000000001e-05 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013827+0000", 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1951e-05 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335974882999999, 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097221809999999999, 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 134 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004214+0000", 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004280+0000", 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5707e-05 2024-09-18T01:02:47.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004291+0000", 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.154e-05 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004306+0000", 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5088e-05 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006681+0000", 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002374953 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006754+0000", 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2490000000000006e-05 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006825+0000", 2024-09-18T01:02:47.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1667000000000003e-05 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009234+0000", 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002408477 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013872+0000", 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0046381599999999997 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013906+0000", 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3878999999999999e-05 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013935+0000", 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9598e-05 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335866141, 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097082049999999993, 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 135 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:47.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004323+0000", 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004328+0000", 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5559999999999998e-06 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004338+0000", 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9699999999999994e-06 2024-09-18T01:02:47.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004344+0000", 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4009999999999996e-06 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007275+0000", 2024-09-18T01:02:47.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002931309 2024-09-18T01:02:47.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007321+0000", 2024-09-18T01:02:47.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6292000000000003e-05 2024-09-18T01:02:47.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007394+0000", 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2354000000000006e-05 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010084+0000", 2024-09-18T01:02:47.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0026903740000000002 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013970+0000", 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0038859210000000001 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013997+0000", 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.669e-05 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014030+0000", 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.345e-05 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.438 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.439 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.439 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.439 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:47.439 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335834812000002, 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097521970000000006, 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.440 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 136 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004354+0000", 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7800000000000001e-07 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004360+0000", 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9660000000000001e-06 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004370+0000", 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0550999999999999e-05 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004374+0000", 2024-09-18T01:02:47.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0500000000000002e-06 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007634+0000", 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003259498 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007684+0000", 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9768000000000002e-05 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007758+0000", 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4017999999999995e-05 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010088+0000", 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0023301569999999998 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014061+0000", 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039732439999999999 2024-09-18T01:02:47.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014087+0000", 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.618e-05 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014105+0000", 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8187000000000001e-05 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335803628000001, 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098164900000000006, 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 137 2024-09-18T01:02:47.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004385+0000", 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004432+0000", 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7184000000000001e-05 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004437+0000", 2024-09-18T01:02:47.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1730000000000003e-06 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004444+0000", 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008052+0000", 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036079279999999998 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008097+0000", 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5011e-05 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008170+0000", 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2904999999999994e-05 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010090+0000", 2024-09-18T01:02:47.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001920274 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014138+0000", 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040482019999999999 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014166+0000", 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7657999999999998e-05 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014201+0000", 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4888000000000002e-05 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335731507999999, 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098169759999999998, 2024-09-18T01:02:47.453 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 138 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004460+0000", 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2499999999999998e-06 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004464+0000", 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3239999999999996e-06 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004467+0000", 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8289999999999999e-06 2024-09-18T01:02:47.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008528+0000", 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040605679999999996 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008574+0000", 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6492999999999997e-05 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008645+0000", 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0403999999999999e-05 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010093+0000", 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001447802 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014230+0000", 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0041371280000000003 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014254+0000", 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4757e-05 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014274+0000", 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9105000000000001e-05 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335716518000002, 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098886330000000008, 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 139 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.460 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004475+0000", 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8720000000000002e-06 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004479+0000", 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1060000000000002e-06 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004482+0000", 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7369999999999998e-06 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008917+0000", 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.004435184 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008966+0000", 2024-09-18T01:02:47.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9592000000000002e-05 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009037+0000", 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0797999999999995e-05 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010095+0000", 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0010581399999999999 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014302+0000", 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042062189999999998 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014328+0000", 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6432e-05 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014360+0000", 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2190000000000002e-05 2024-09-18T01:02:47.465 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335702925, 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.012201405, 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.466 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 140 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-07 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004529+0000", 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3432e-05 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004533+0000", 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5249999999999999e-06 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004537+0000", 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8160000000000004e-06 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009263+0000", 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0047261509999999996 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009311+0000", 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7568000000000003e-05 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009381+0000", 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0419000000000005e-05 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010098+0000", 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00071670299999999996 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016625+0000", 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0065267939999999998 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016651+0000", 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5959999999999999e-05 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016687+0000", 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5772000000000002e-05 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 20.335645533000001, 2024-09-18T01:02:47.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.012284938, 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 141 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:47.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2300000000000002e-07 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004546+0000", 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.864e-06 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004552+0000", 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1999999999999999e-06 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004555+0000", 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.706e-06 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009757+0000", 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052024180000000003 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009806+0000", 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8653999999999999e-05 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009879+0000", 2024-09-18T01:02:47.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3551000000000001e-05 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010335+0000", 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00045522899999999997 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016767+0000", 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0064324079999999997 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016790+0000", 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3468999999999998e-05 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016828+0000", 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7116000000000002e-05 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.479 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:47.482 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:02:47.482 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:47.479+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:47.482 INFO:teuthology.orchestra.run.smithi164.stdout:269 trunc from 0xbdb5600 to 0x82c0a00 2024-09-18T01:02:47.482 INFO:teuthology.orchestra.run.smithi164.stdout:270 write 0xed88a00 thru 0xed96bdf (0xe1e0 bytes) 2024-09-18T01:02:47.482 INFO:teuthology.orchestra.run.smithi164.stdout:271 trunc from 0xed96c00 to 0x184f000 2024-09-18T01:02:47.482 INFO:teuthology.orchestra.run.smithi164.stdout:272 punch from 0x17b0a00 to 0x17bc000, (0xb600 bytes) 2024-09-18T01:02:47.482 INFO:teuthology.orchestra.run.smithi164.stdout:273 read 0xf7bc00 thru 0xf896cd (0xdace bytes) 2024-09-18T01:02:47.483 INFO:teuthology.orchestra.run.smithi164.stdout:274 trunc from 0x184f000 to 0xb5d400 2024-09-18T01:02:47.483 INFO:teuthology.orchestra.run.smithi164.stdout:275 trunc from 0xb5d400 to 0x1e56800 2024-09-18T01:02:47.485 INFO:teuthology.orchestra.run.smithi164.stdout:278 write 0xa45e00 thru 0xa48320 (0x2521 bytes) 2024-09-18T01:02:47.486 INFO:teuthology.orchestra.run.smithi164.stdout:282 read 0x120ce00 thru 0x121a8d9 (0xdada bytes) 2024-09-18T01:02:47.486 INFO:teuthology.orchestra.run.smithi164.stdout:283 read 0xc5d000 thru 0xc6ad16 (0xdd17 bytes) 2024-09-18T01:02:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:02:47.487 INFO:teuthology.orchestra.run.smithi164.stdout:286 punch from 0xd9de00 to 0xda7400, (0x9600 bytes) 2024-09-18T01:02:47.511 INFO:teuthology.orchestra.run.smithi164.stdout:287 write 0x7dc6600 thru 0x7dd4e87 (0xe888 bytes) 2024-09-18T01:02:47.511 INFO:teuthology.orchestra.run.smithi164.stdout:288 read 0x7534400 thru 0x7537671 (0x3272 bytes) 2024-09-18T01:02:47.512 INFO:teuthology.orchestra.run.smithi164.stdout:293 write 0x7314c00 thru 0x7318639 (0x3a3a bytes) 2024-09-18T01:02:47.512 INFO:teuthology.orchestra.run.smithi164.stdout:294 write 0x360a800 thru 0x360fe77 (0x5678 bytes) 2024-09-18T01:02:47.539 INFO:teuthology.orchestra.run.smithi164.stdout:296 write 0x988c200 thru 0x9897745 (0xb546 bytes) 2024-09-18T01:02:47.539 INFO:teuthology.orchestra.run.smithi164.stdout:297 read 0x28f9400 thru 0x29021aa (0x8dab bytes) 2024-09-18T01:02:47.540 INFO:teuthology.orchestra.run.smithi164.stdout:298 write 0x6f27800 thru 0x6f2e6ae (0x6eaf bytes) 2024-09-18T01:02:47.540 INFO:teuthology.orchestra.run.smithi164.stdout:299 read 0x2608e00 thru 0x2617e9a (0xf09b bytes) 2024-09-18T01:02:47.540 INFO:teuthology.orchestra.run.smithi164.stdout:301 read 0x6c67800 thru 0x6c6e19c (0x699d bytes) 2024-09-18T01:02:47.565 INFO:teuthology.orchestra.run.smithi164.stdout:302 write 0xae21400 thru 0xae2abd2 (0x97d3 bytes) 2024-09-18T01:02:47.565 INFO:teuthology.orchestra.run.smithi164.stdout:308 read 0x31b2200 thru 0x31b940e (0x720f bytes) 2024-09-18T01:02:47.565 INFO:teuthology.orchestra.run.smithi164.stdout:309 write 0x645b800 thru 0x645cf38 (0x1739 bytes) 2024-09-18T01:02:47.565 INFO:teuthology.orchestra.run.smithi164.stdout:310 flatten 2024-09-18T01:02:47.581 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:47.580+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:47.584 INFO:teuthology.orchestra.run.smithi164.stdout:312 trunc from 0xae2ac00 to 0x94f3c00 2024-09-18T01:02:47.607 INFO:teuthology.orchestra.run.smithi164.stdout:313 write 0x5105600 thru 0x510fe17 (0xa818 bytes) 2024-09-18T01:02:47.607 INFO:teuthology.orchestra.run.smithi164.stdout:314 read 0x375c800 thru 0x376b60d (0xee0e bytes) 2024-09-18T01:02:47.613 INFO:teuthology.orchestra.run.smithi164.stdout:315 trunc from 0x94f3c00 to 0x4729e00 2024-09-18T01:02:47.614 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-18T01:02:47.613+0000 7fdf07c7e640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-18T01:02:47.631 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:02:47.632 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:47.639 INFO:teuthology.orchestra.run.smithi164.stdout:317 read 0x31bb400 thru 0x31cab4e (0xf74f bytes) 2024-09-18T01:02:47.641 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:02:47.657 INFO:teuthology.orchestra.run.smithi164.stdout:318 write 0x56a8000 thru 0x56b5440 (0xd441 bytes) 2024-09-18T01:02:47.660 INFO:teuthology.orchestra.run.smithi164.stdout:319 trunc from 0x56b5600 to 0x275c200 2024-09-18T01:02:47.682 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:47.681+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:47.704 INFO:teuthology.orchestra.run.smithi164.stdout:320 write 0xe50b200 thru 0xe5151cc (0x9fcd bytes) 2024-09-18T01:02:47.704 INFO:teuthology.orchestra.run.smithi164.stdout:321 read 0x29b7e00 thru 0x29b874e (0x94f bytes) 2024-09-18T01:02:47.705 INFO:teuthology.orchestra.run.smithi164.stdout:322 read 0xb693400 thru 0xb6948fc (0x14fd bytes) 2024-09-18T01:02:47.705 INFO:teuthology.orchestra.run.smithi164.stdout:324 punch from 0x8b98e00 to 0x8ba7000, (0xe200 bytes) 2024-09-18T01:02:47.705 INFO:teuthology.orchestra.run.smithi164.stdout:326 punch from 0x9f9600 to 0x9fae00, (0x1800 bytes) 2024-09-18T01:02:47.705 INFO:teuthology.orchestra.run.smithi164.stdout:327 write 0x6ad3200 thru 0x6adbca5 (0x8aa6 bytes) 2024-09-18T01:02:47.705 INFO:teuthology.orchestra.run.smithi164.stdout:328 read 0x43f2c00 thru 0x43fe329 (0xb72a bytes) 2024-09-18T01:02:47.708 INFO:teuthology.orchestra.run.smithi164.stdout:331 trunc from 0xe515200 to 0xc26fa00 2024-09-18T01:02:47.725 INFO:teuthology.orchestra.run.smithi164.stdout:333 write 0x1e50200 thru 0x1e5331f (0x3120 bytes) 2024-09-18T01:02:47.732 INFO:teuthology.orchestra.run.smithi164.stdout:334 trunc from 0xc26fa00 to 0x70a9e00 2024-09-18T01:02:47.748 INFO:teuthology.orchestra.run.smithi164.stdout:335 read 0x3ad2e00 thru 0x3ad5a5a (0x2c5b bytes) 2024-09-18T01:02:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:47.749 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:47.758 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:02:47.758 INFO:teuthology.orchestra.run.smithi164.stdout:337 write 0x79abe00 thru 0x79b7988 (0xbb89 bytes) 2024-09-18T01:02:47.765 INFO:teuthology.orchestra.run.smithi164.stdout:338 clone 2 order 24 su 8388608 sc 13 2024-09-18T01:02:47.783 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:47.782+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.470223+0000", 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.3895939359999998, 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0061150029999999999, 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.862 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 9 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470223+0000", 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470223+0000", 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470224+0000", 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3349999999999999e-06 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470237+0000", 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3030999999999999e-05 2024-09-18T01:02:47.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470248+0000", 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1022999999999999e-05 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470255+0000", 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2710000000000003e-06 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470300+0000", 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4777999999999997e-05 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470307+0000", 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4160000000000001e-06 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.473645+0000", 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033382479999999998 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.473713+0000", 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8299000000000005e-05 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.473798+0000", 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5017000000000003e-05 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.474959+0000", 2024-09-18T01:02:47.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0011607900000000001 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.476275+0000", 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013163700000000001 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.476311+0000", 2024-09-18T01:02:47.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6127999999999997e-05 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.476338+0000", 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6296999999999999e-05 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4355.0:10 3.1c e23/20 3:3831950c:::rbd_directory:head v 23'1, mlcod=0'0)", 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.479726+0000", 2024-09-18T01:02:47.869 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.3800901650000004, 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025737210000000002, 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479726+0000", 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479726+0000", 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479727+0000", 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.74e-07 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479730+0000", 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2849999999999999e-06 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479740+0000", 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0088000000000001e-05 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479748+0000", 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5499999999999997e-06 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479793+0000", 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5195999999999999e-05 2024-09-18T01:02:47.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479811+0000", 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8668000000000002e-05 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482259+0000", 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00244765 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482300+0000", 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1109999999999998e-05 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:720 3.3 3:c542c01e:::rbd_data.1103b05c2cf1.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.428965+0000", 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.430851188, 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0048215059999999997, 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.874 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 720 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428965+0000", 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428965+0000", 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428966+0000", 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428971+0000", 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2220000000000004e-06 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428983+0000", 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2877999999999999e-05 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428990+0000", 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9650000000000002e-06 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429044+0000", 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3310999999999998e-05 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429124+0000", 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0663999999999999e-05 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429262+0000", 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000138138 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432562+0000", 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032997880000000001 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.433718+0000", 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0011555090000000001 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.433750+0000", 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1831000000000001e-05 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.433787+0000", 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7385999999999997e-05 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:710 3.1b e26/20 3:d93a887d:::rbd_data.1103b05c2cf1.000000000000001d:head v 26'87, mlcod=26'85)", 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429361+0000", 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.430455488, 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032690079999999999, 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429361+0000", 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429361+0000", 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429362+0000", 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429367+0000", 2024-09-18T01:02:47.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1869999999999996e-06 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429377+0000", 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9410000000000002e-06 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429385+0000", 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7030000000000006e-06 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.883 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429424+0000", 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9208000000000001e-05 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429446+0000", 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1878e-05 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432590+0000", 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031443370000000001 2024-09-18T01:02:47.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432630+0000", 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0147e-05 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:719 3.1 e26/20 3:855394e7:::rbd_data.1103b05c2cf1.0000000000000023:head v 26'6, mlcod=26'4)", 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.43041770899999998, 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003258295, 2024-09-18T01:02:47.885 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0100000000000001e-07 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429403+0000", 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3110000000000001e-06 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429410+0000", 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429414+0000", 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1500000000000001e-06 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429471+0000", 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6928000000000001e-05 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429490+0000", 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8627e-05 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432606+0000", 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031164560000000001 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432657+0000", 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1102e-05 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:714 3.19 e26/20 3:9c24ad54:::rbd_data.1103b05c2cf1.0000000000000020:head v 26'101, mlcod=26'99)", 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429597+0000", 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.43021943200000001, 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0030934769999999999, 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.890 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429597+0000", 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429597+0000", 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429598+0000", 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6700000000000003e-07 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429604+0000", 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1360000000000002e-06 2024-09-18T01:02:47.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429614+0000", 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5529999999999998e-06 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429620+0000", 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9859999999999999e-06 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429665+0000", 2024-09-18T01:02:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5389999999999997e-05 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429683+0000", 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8056000000000001e-05 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432659+0000", 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002976228 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432691+0000", 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1461000000000003e-05 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:758 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'97, mlcod=26'88)", 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.444668+0000", 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.415148562, 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002755633, 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.895 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444668+0000", 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444668+0000", 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444669+0000", 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.13e-07 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444673+0000", 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7509999999999999e-06 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444680+0000", 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2840000000000003e-06 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444684+0000", 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1910000000000002e-06 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444712+0000", 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7784999999999999e-05 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.444723+0000", 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1018e-05 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447387+0000", 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0026647020000000001 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447424+0000", 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6389e-05 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:760 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'98, mlcod=26'88)", 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.445182+0000", 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.41463466700000001, 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050740170000000001, 2024-09-18T01:02:47.900 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445182+0000", 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445182+0000", 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445183+0000", 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2799999999999996e-07 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445186+0000", 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4130000000000002e-06 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445193+0000", 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9469999999999997e-06 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445197+0000", 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7220000000000001e-06 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445268+0000", 2024-09-18T01:02:47.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1180000000000006e-05 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445286+0000", 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8346e-05 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450215+0000", 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049289410000000001 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450256+0000", 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0939999999999998e-05 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:761 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'99, mlcod=26'88)", 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.41381401400000001, 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.004282358, 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.907 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:47.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2500000000000001e-07 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446006+0000", 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446012+0000", 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1020000000000002e-06 2024-09-18T01:02:47.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446016+0000", 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2200000000000001e-06 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446049+0000", 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3358000000000001e-05 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446063+0000", 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3813e-05 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450258+0000", 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0041955600000000001 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450285+0000", 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6588999999999999e-05 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.911 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:762 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'100, mlcod=26'88)", 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.41314089100000001, 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036332729999999998, 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.912 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.913 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8000000000000002e-07 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446679+0000", 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446684+0000", 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2650000000000004e-06 2024-09-18T01:02:47.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446687+0000", 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.88e-06 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446732+0000", 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5349000000000002e-05 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446751+0000", 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8719000000000002e-05 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450286+0000", 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0035353849999999998 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450309+0000", 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2874000000000002e-05 2024-09-18T01:02:47.916 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:765 3.1d e26/20 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head v 26'3, mlcod=26'1)", 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447708+0000", 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.41210866200000001, 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052494550000000001, 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.917 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447708+0000", 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447708+0000", 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447709+0000", 2024-09-18T01:02:47.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447715+0000", 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2360000000000001e-06 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447727+0000", 2024-09-18T01:02:47.919 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1894e-05 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447734+0000", 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.002e-06 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447786+0000", 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2216999999999998e-05 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.920 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447808+0000", 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1682e-05 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452916+0000", 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051077450000000003 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452957+0000", 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1865e-05 2024-09-18T01:02:47.921 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:763 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'101, mlcod=26'95)", 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.41188004499999997, 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.005068571, 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.922 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1500000000000004e-07 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447946+0000", 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2819999999999996e-06 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447956+0000", 2024-09-18T01:02:47.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9590000000000007e-06 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447962+0000", 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448069+0000", 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010698 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448087+0000", 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8104000000000001e-05 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452970+0000", 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0048830820000000004 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453005+0000", 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4807000000000001e-05 2024-09-18T01:02:47.926 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:770 3.1 e26/20 3:85d4f36e:::rbd_data.1103b05c2cf1.0000000000000014:head v 26'8, mlcod=26'6)", 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447979+0000", 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.41183748599999997, 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050070510000000002, 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447979+0000", 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447979+0000", 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447980+0000", 2024-09-18T01:02:47.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447985+0000", 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447992+0000", 2024-09-18T01:02:47.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0960000000000003e-06 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447997+0000", 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9849999999999997e-06 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448045+0000", 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8541000000000002e-05 2024-09-18T01:02:47.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448063+0000", 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8084000000000002e-05 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452948+0000", 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0048849430000000001 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452986+0000", 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7858000000000002e-05 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:773 3.3 3:c65d1f6d:::rbd_data.1103b05c2cf1.0000000000000017:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.450741+0000", 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.40907561100000001, 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0044119839999999999, 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.932 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 773 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450741+0000", 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450741+0000", 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450742+0000", 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0279999999999999e-06 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.934 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450747+0000", 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8369999999999996e-06 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450757+0000", 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0043e-05 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450762+0000", 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450807+0000", 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4376999999999999e-05 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450884+0000", 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7440000000000004e-05 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450996+0000", 2024-09-18T01:02:47.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111579 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453291+0000", 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0022950750000000002 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.455096+0000", 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0018052980000000001 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.455127+0000", 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0642999999999998e-05 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.455153+0000", 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6429000000000001e-05 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:774 3.1a 3:5f37e58f:::rbd_data.1103b05c2cf1.0000000000000018:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:47.938 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.450775+0000", 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.40904194100000002, 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025993259999999999, 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 774 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450775+0000", 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450775+0000", 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450775+0000", 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1e-07 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450777+0000", 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450786+0000", 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6330000000000002e-06 2024-09-18T01:02:47.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450790+0000", 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3239999999999996e-06 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450828+0000", 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8139999999999997e-05 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450910+0000", 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2211000000000003e-05 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.451024+0000", 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000113539 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452262+0000", 2024-09-18T01:02:47.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012386159999999999 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453313+0000", 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0010508959999999999 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453348+0000", 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4922999999999998e-05 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453374+0000", 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5772e-05 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.945 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:830 3.1a 3:5f37e58f:::rbd_data.1103b05c2cf1.0000000000000018:head [stat,write 515584~8704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.487225+0000", 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.37259189999999998, 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.007291563, 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 830 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.946 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487225+0000", 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487225+0000", 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487226+0000", 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.54e-07 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487247+0000", 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1080000000000001e-05 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487259+0000", 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.201e-05 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487264+0000", 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2889999999999997e-06 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487317+0000", 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3007999999999998e-05 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487323+0000", 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0209999999999996e-06 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.489522+0000", 2024-09-18T01:02:47.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002198565 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.489574+0000", 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1956000000000002e-05 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.489715+0000", 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014140999999999999 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.490363+0000", 2024-09-18T01:02:47.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064768300000000003 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.494457+0000", 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040941229999999999 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.494490+0000", 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2891000000000001e-05 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.494516+0000", 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6673e-05 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:851 3.1b e27/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 27'107, mlcod=26'103)", 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.515887+0000", 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.343929504, 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0022351150000000002, 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.952 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515887+0000", 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515887+0000", 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515888+0000", 2024-09-18T01:02:47.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5700000000000002e-07 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515892+0000", 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4120000000000003e-06 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515902+0000", 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.6909999999999992e-06 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515906+0000", 2024-09-18T01:02:47.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6670000000000004e-06 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515951+0000", 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4802000000000002e-05 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.515970+0000", 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9143e-05 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518083+0000", 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021130039999999999 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518122+0000", 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8838999999999999e-05 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:854 3.3 3:c33c4291:::rbd_data.1103b05c2cf1.00000000000000fa:head [set-alloc-hint object_size 524288 write_size 524288,write 418816~60416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.518391+0000", 2024-09-18T01:02:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.34142592199999999, 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025496999999999998, 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 854 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518391+0000", 2024-09-18T01:02:47.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518391+0000", 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518391+0000", 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1999999999999999e-07 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518466+0000", 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4198999999999999e-05 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518476+0000", 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0037999999999999e-05 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518481+0000", 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518528+0000", 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7369e-05 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518652+0000", 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012381499999999999 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.518792+0000", 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014047500000000001 2024-09-18T01:02:47.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.519602+0000", 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00080946800000000004 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.520845+0000", 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012434169999999999 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.520896+0000", 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0819000000000003e-05 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.520940+0000", 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4350000000000001e-05 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:873 3.1b e27/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 27'110, mlcod=27'107)", 2024-09-18T01:02:47.962 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.542932+0000", 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.31688435999999998, 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0028678589999999999, 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.542932+0000", 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.542932+0000", 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.542933+0000", 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2e-07 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.542939+0000", 2024-09-18T01:02:47.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0589999999999999e-06 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.542950+0000", 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1260000000000001e-05 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.542955+0000", 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9089999999999999e-06 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.543004+0000", 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8520999999999998e-05 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.543020+0000", 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6379999999999999e-05 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.545767+0000", 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002747443 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.545800+0000", 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2767e-05 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:961 3.3 3:c33c4291:::rbd_data.1103b05c2cf1.00000000000000fa:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.620802+0000", 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.23901428799999999, 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0045840669999999998, 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 961 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620802+0000", 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620802+0000", 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620804+0000", 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2810000000000001e-06 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620808+0000", 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4299999999999999e-06 2024-09-18T01:02:47.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620822+0000", 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4287000000000001e-05 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620834+0000", 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1747e-05 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620889+0000", 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4425000000000002e-05 2024-09-18T01:02:47.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620977+0000", 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.8243000000000002e-05 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.621088+0000", 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111494 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.621657+0000", 2024-09-18T01:02:47.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00056824299999999996 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.625295+0000", 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036382680000000001 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.625347+0000", 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1882999999999998e-05 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:47.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.625386+0000", 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9765999999999998e-05 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:47.973 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:47.974 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:47.973+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:47.975 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:02:48.076 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:48.075+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:48.079 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:48.079 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:48.079 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:02:48.079 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:48.087 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:02:48.177 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:48.176+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:48.189 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:48.190 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:48.198 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:02:48.276 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.5. 2024-09-18T01:02:48.279 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:48.277+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:48.347 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.970946924, 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:02:48.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:02:48.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:02:48.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:02:48.354 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 14.972935618999999, 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:02:48.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:02:48.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:02:48.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.965258667000001, 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:48.362 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:02:48.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:02:48.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.368 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.8674007169999998, 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.369 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:02:48.375 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 1.011319705, 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.376 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:02:48.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:02:48.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:02:48.382 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 1.0109504090000001, 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:02:48.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:02:48.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 1.0092115639999999, 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:02:48.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:02:48.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:02:48.393 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 1.009139674, 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.397 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.92127366399999999, 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.398 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:02:48.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.91508240500000004, 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.404 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:02:48.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:02:48.408 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.91355271999999998, 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:02:48.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:02:48.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.91317303100000002, 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.415 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:02:48.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:02:48.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:02:48.421 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.89767462799999997, 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:02:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:02:48.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.427 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.89746786899999997, 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:02:48.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.433 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.434 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.434 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:02:48.434 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:02:48.434 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.89705011199999996, 2024-09-18T01:02:48.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:02:48.435 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:02:48.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.89684087400000001, 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.440 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:02:48.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:02:48.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.89677693999999997, 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:02:48.445 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:02:48.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.89602873500000002, 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.450 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.63708993000000003, 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:02:48.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:02:48.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:02:48.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.63622631200000002, 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:48.461 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:48.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:48.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:48.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:02:48.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:02:48.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:02:48.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.467 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:48.467 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:48.467 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:48.468 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:48.467+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:48.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:48.569 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:48.568+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:48.585 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:48.585 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:48.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:48.670 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:48.669+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:48.741 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:48.741 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:48.750 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:48.794 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:48.793+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:48.859 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:130 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'88, mlcod=13'85)", 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.852409167000001, 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085906969999999996, 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004681+0000", 2024-09-18T01:02:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5499999999999996e-07 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004687+0000", 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7459999999999997e-06 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004697+0000", 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0513000000000001e-05 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004703+0000", 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004741+0000", 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7551e-05 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004751+0000", 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0497e-05 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013227+0000", 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084754370000000006 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013271+0000", 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4280000000000003e-05 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.851884926, 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080847769999999996, 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005205+0000", 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-07 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005281+0000", 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5927999999999995e-05 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005290+0000", 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5920000000000007e-06 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005295+0000", 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.605e-06 2024-09-18T01:02:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005342+0000", 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7327000000000003e-05 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005364+0000", 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1583999999999999e-05 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013275+0000", 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007910594 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013289+0000", 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4561e-05 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.85134966, 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075705210000000002, 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005740+0000", 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3200000000000005e-07 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005746+0000", 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4090000000000002e-06 2024-09-18T01:02:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005756+0000", 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0342e-05 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005761+0000", 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005814+0000", 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3603999999999997e-05 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005827+0000", 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2931e-05 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013291+0000", 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074636470000000003 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013310+0000", 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9211999999999998e-05 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'86)", 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.006445+0000", 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.850644022000001, 2024-09-18T01:02:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068845750000000004, 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006445+0000", 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006445+0000", 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006446+0000", 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3e-07 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006450+0000", 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4190000000000004e-06 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006460+0000", 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0162e-05 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006465+0000", 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006502+0000", 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6693999999999998e-05 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.006514+0000", 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1541e-05 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013314+0000", 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067999519999999997 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013330+0000", 2024-09-18T01:02:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6303999999999998e-05 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'86)", 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.847605623, 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069265170000000001, 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009484+0000", 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0900000000000001e-07 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009528+0000", 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4047e-05 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009539+0000", 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0444e-05 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.009543+0000", 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6439999999999999e-06 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010886+0000", 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001342244 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.010899+0000", 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2965e-05 2024-09-18T01:02:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.016373+0000", 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054748009999999996 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.016410+0000", 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6662999999999998e-05 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:501 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.261683+0000", 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5954061990000001, 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007647198, 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 501 2024-09-18T01:02:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261683+0000", 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261683+0000", 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261684+0000", 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0890000000000001e-06 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261690+0000", 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7300000000000002e-06 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261702+0000", 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2512e-05 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261710+0000", 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0080000000000002e-06 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263324+0000", 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016138400000000001 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263417+0000", 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2987999999999998e-05 2024-09-18T01:02:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263753+0000", 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033545200000000003 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.267736+0000", 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039828939999999998 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.269260+0000", 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015246669999999999 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.269298+0000", 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8204000000000002e-05 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.269330+0000", 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1813999999999997e-05 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:502 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.262195+0000", 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.594894335, 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072415200000000004, 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 502 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262195+0000", 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262195+0000", 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262196+0000", 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6099999999999999e-07 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262201+0000", 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9189999999999998e-06 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262212+0000", 2024-09-18T01:02:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090000000000001e-05 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262217+0000", 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.264144+0000", 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019270209999999999 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.264227+0000", 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2993999999999999e-05 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.264478+0000", 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025157000000000001 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.267739+0000", 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032612159999999999 2024-09-18T01:02:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.269364+0000", 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001624515 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.269398+0000", 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3612000000000003e-05 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.269436+0000", 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8825999999999997e-05 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:751 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440137+0000", 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.416952161, 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068108739999999997, 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 751 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440137+0000", 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440137+0000", 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440138+0000", 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440140+0000", 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5979999999999999e-06 2024-09-18T01:02:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440145+0000", 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1919999999999996e-06 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440149+0000", 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5250000000000001e-06 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440265+0000", 2024-09-18T01:02:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011616399999999999 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440307+0000", 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2117999999999999e-05 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440490+0000", 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000182428 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443199+0000", 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027090959999999998 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446896+0000", 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003697274 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446924+0000", 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7518999999999999e-05 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446948+0000", 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4358e-05 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:752 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440155+0000", 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4169346119999999, 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068396350000000002, 2024-09-18T01:02:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 752 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440155+0000", 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440155+0000", 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440155+0000", 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440157+0000", 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5319999999999999e-06 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440161+0000", 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0579999999999999e-06 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440163+0000", 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.683e-06 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440866+0000", 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070259000000000001 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440914+0000", 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7753999999999997e-05 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441134+0000", 2024-09-18T01:02:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000220189 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443241+0000", 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021066689999999998 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446953+0000", 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037129720000000001 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446976+0000", 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2710000000000001e-05 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446994+0000", 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8090000000000001e-05 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:753 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440167+0000", 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.416922171, 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068747599999999997, 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 753 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440167+0000", 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440167+0000", 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440167+0000", 2024-09-18T01:02:48.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440169+0000", 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.327e-06 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440172+0000", 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1420000000000002e-06 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440175+0000", 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.106e-06 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441387+0000", 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012125969999999999 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441444+0000", 2024-09-18T01:02:48.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489000000000001e-05 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441744+0000", 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030017500000000002 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444276+0000", 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025318390000000001 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446999+0000", 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027227649999999998 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447020+0000", 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1194000000000001e-05 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447042+0000", 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1916000000000002e-05 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:754 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440178+0000", 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4169110630000001, 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069070759999999998, 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 754 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440178+0000", 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440178+0000", 2024-09-18T01:02:48.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440178+0000", 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440186+0000", 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7519999999999996e-06 2024-09-18T01:02:48.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440190+0000", 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-06 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440194+0000", 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4229999999999998e-06 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.442027+0000", 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018333030000000001 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.442074+0000", 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6561999999999999e-05 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.442255+0000", 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018113100000000001 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444309+0000", 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002054368 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447046+0000", 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027370070000000001 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447069+0000", 2024-09-18T01:02:48.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2189999999999999e-05 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447085+0000", 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6718000000000002e-05 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:755 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440201+0000", 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4168880420000001, 2024-09-18T01:02:48.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069266579999999996, 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 755 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440201+0000", 2024-09-18T01:02:48.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440201+0000", 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440202+0000", 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440203+0000", 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5600000000000001e-06 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440207+0000", 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4979999999999998e-06 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440210+0000", 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.442518+0000", 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002308325 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.442564+0000", 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6156999999999999e-05 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.442767+0000", 2024-09-18T01:02:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020321900000000001 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444345+0000", 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015776989999999999 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447090+0000", 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027448379999999999 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447111+0000", 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0755e-05 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447128+0000", 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7343999999999999e-05 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:756 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440213+0000", 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.416876236, 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069783800000000002, 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 756 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440213+0000", 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440213+0000", 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440213+0000", 2024-09-18T01:02:48.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.35e-07 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440215+0000", 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5340000000000001e-06 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440218+0000", 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0259999999999999e-06 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440220+0000", 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.441e-06 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443274+0000", 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003053385 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443330+0000", 2024-09-18T01:02:48.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489000000000001e-05 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443515+0000", 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184425 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445389+0000", 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018741890000000001 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447136+0000", 2024-09-18T01:02:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017467590000000001 2024-09-18T01:02:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447171+0000", 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5373999999999997e-05 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447191+0000", 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0523e-05 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:757 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440223+0000", 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4168658760000001, 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070184000000000002, 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 757 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440223+0000", 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440223+0000", 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440224+0000", 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440227+0000", 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5599999999999998e-06 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440230+0000", 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1640000000000001e-06 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440234+0000", 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7799999999999998e-06 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443791+0000", 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003556717 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443838+0000", 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7373999999999997e-05 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444020+0000", 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000181733 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445428+0000", 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014077040000000001 2024-09-18T01:02:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447196+0000", 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017687029999999999 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1399999999999998e-05 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447242+0000", 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4059000000000001e-05 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:758 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440237+0000", 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.416851844, 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079948129999999999, 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 758 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440237+0000", 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440237+0000", 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440238+0000", 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.11e-07 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440239+0000", 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440242+0000", 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0469999999999998e-06 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440246+0000", 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1110000000000001e-06 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444380+0000", 2024-09-18T01:02:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041344700000000003 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444428+0000", 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7969000000000001e-05 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444610+0000", 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018195799999999999 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447248+0000", 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026375719999999999 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448162+0000", 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091418599999999999 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448204+0000", 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1946999999999997e-05 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448232+0000", 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8102e-05 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:760 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4168400649999999, 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01021648, 2024-09-18T01:02:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 760 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440251+0000", 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2979999999999999e-06 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440254+0000", 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9409999999999999e-06 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440256+0000", 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5040000000000001e-06 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444891+0000", 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046348149999999996 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444937+0000", 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6516e-05 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445121+0000", 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018370600000000001 2024-09-18T01:02:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021303839999999999 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450406+0000", 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031547519999999998 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450440+0000", 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3844999999999999e-05 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450466+0000", 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5517999999999999e-05 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:761 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.416830115, 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010302318, 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 761 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440261+0000", 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.469e-06 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440264+0000", 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9579999999999999e-06 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440266+0000", 2024-09-18T01:02:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.508e-06 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445464+0000", 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051974660000000004 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445536+0000", 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1999e-05 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445930+0000", 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039429200000000003 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447255+0000", 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013246810000000001 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450506+0000", 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032513210000000002 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450536+0000", 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0437e-05 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450561+0000", 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4983000000000001e-05 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:762 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4168200419999999, 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010367795000000001, 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 762 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440271+0000", 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440274+0000", 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.943e-06 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440276+0000", 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446351+0000", 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060752339999999997 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446400+0000", 2024-09-18T01:02:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8437999999999999e-05 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446610+0000", 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000209772 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448140+0000", 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001530214 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450587+0000", 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024474010000000001 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450614+0000", 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6769999999999999e-05 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450637+0000", 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3110000000000001e-05 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:763 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4168103809999999, 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012961914, 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 763 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9400000000000001e-07 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440280+0000", 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1829999999999999e-06 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440283+0000", 2024-09-18T01:02:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.931e-06 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440286+0000", 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447287+0000", 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070015650000000004 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447380+0000", 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2727999999999997e-05 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447750+0000", 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036956400000000001 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.452379+0000", 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046289130000000001 2024-09-18T01:02:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453138+0000", 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075947999999999999 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453196+0000", 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8050999999999997e-05 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453241+0000", 2024-09-18T01:02:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4626000000000003e-05 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:829 3.1f 3:fdd092e6:::rbd_data.1103b05c2cf1.000000000000001a:head [stat,write 154624~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.487086+0000", 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3700035850000001, 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084065200000000007, 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 829 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487086+0000", 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487086+0000", 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487087+0000", 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7000000000000003e-07 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487102+0000", 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5900999999999999e-05 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487113+0000", 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1045e-05 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487119+0000", 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3750000000000002e-06 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487174+0000", 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5621000000000002e-05 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487183+0000", 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0979999999999994e-06 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.492940+0000", 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057575730000000002 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493032+0000", 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1692000000000002e-05 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493457+0000", 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042539200000000002 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.494574+0000", 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011166049999999999 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495381+0000", 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080743899999999996 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495442+0000", 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0958999999999997e-05 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495492+0000", 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9950000000000001e-05 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:48.979 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:48.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:02:48.981 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:48.979+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:49.081 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:49.080+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:49.128 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3.9a90f902 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.85046790500000002, 2024-09-18T01:02:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.85049010999999997, 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:02:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3.9a90f902 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.84584200200000004, 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.84591240000000001, 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:02:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:02:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3.9a90f902 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.84576395299999996, 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.84587512899999995, 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:02:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:02:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 3 2024-09-18T01:02:49.144 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:49.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:02:49.182 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:49.181+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:49.251 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:49.251 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:49.251 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:49.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:02:49.284 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:49.282+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:49.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:02:49.384 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:49.383+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:458 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.257421+0000", 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1332748829999999, 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040566500000000002, 2024-09-18T01:02:49.393 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 458 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257421+0000", 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257421+0000", 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257422+0000", 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9300000000000006e-07 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257427+0000", 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2700000000000004e-06 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257440+0000", 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2446000000000001e-05 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257447+0000", 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4789999999999999e-06 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261302+0000", 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038547569999999999 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261376+0000", 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3753999999999999e-05 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261478+0000", 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101951 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:459 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.257668+0000", 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1330283290000001, 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040078739999999998, 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 459 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257668+0000", 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257668+0000", 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257668+0000", 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.47e-07 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257673+0000", 2024-09-18T01:02:49.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257692+0000", 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8780999999999999e-05 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257700+0000", 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0269999999999995e-06 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261526+0000", 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038258849999999998 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261600+0000", 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4517999999999994e-05 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261675+0000", 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5148999999999995e-05 2024-09-18T01:02:49.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:771 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.257727+0000", 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1329691639999999, 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041119729999999997, 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 771 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257727+0000", 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257727+0000", 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257728+0000", 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5899999999999995e-07 2024-09-18T01:02:49.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257732+0000", 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.78e-06 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257744+0000", 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1259e-05 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257752+0000", 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1100000000000003e-06 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261707+0000", 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039551129999999997 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261770+0000", 2024-09-18T01:02:49.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3398000000000005e-05 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261839+0000", 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8454000000000006e-05 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:460 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.257937+0000", 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1327592109999998, 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042138870000000004, 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 460 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257937+0000", 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257937+0000", 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257938+0000", 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257942+0000", 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-18T01:02:49.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257954+0000", 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1883e-05 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257970+0000", 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6277999999999999e-05 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262010+0000", 2024-09-18T01:02:49.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040396169999999997 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262046+0000", 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6381000000000003e-05 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262151+0000", 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010419799999999999 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:772 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.257953+0000", 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1327427669999999, 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040249580000000004, 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 772 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257953+0000", 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257953+0000", 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257954+0000", 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7800000000000001e-07 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257957+0000", 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.568e-06 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257965+0000", 2024-09-18T01:02:49.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4070000000000004e-06 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257969+0000", 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3329999999999998e-06 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261858+0000", 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038892639999999999 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261915+0000", 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6616999999999997e-05 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261978+0000", 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990999999999994e-05 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:773 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258077+0000", 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1326188579999998, 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042906730000000001, 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 773 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258077+0000", 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258077+0000", 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258078+0000", 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1399999999999996e-07 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258082+0000", 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4830000000000001e-06 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258093+0000", 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0861e-05 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258100+0000", 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6800000000000004e-06 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262199+0000", 2024-09-18T01:02:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040993239999999997 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262280+0000", 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1094999999999995e-05 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262368+0000", 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7515999999999994e-05 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:461 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258099+0000", 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1325973239999998, 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044326670000000004, 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 461 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258099+0000", 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258099+0000", 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258100+0000", 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.071e-06 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258104+0000", 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3939999999999998e-06 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258114+0000", 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0225e-05 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258129+0000", 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4941e-05 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262396+0000", 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042670670000000003 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262433+0000", 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6616000000000003e-05 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262531+0000", 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8352999999999998e-05 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:463 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.428 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1324587909999999, 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045481109999999996, 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 463 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258238+0000", 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4900000000000005e-07 2024-09-18T01:02:49.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258242+0000", 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.138e-06 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258252+0000", 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.005e-05 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258261+0000", 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8279999999999992e-06 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262585+0000", 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043244470000000004 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262670+0000", 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4451999999999997e-05 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262785+0000", 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011534700000000001 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.433 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:774 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.132448138, 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047854200000000003, 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 774 2024-09-18T01:02:49.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258249+0000", 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7700000000000003e-07 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258252+0000", 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999998e-06 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258260+0000", 2024-09-18T01:02:49.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6750000000000002e-06 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258264+0000", 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2150000000000003e-06 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262838+0000", 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045744319999999998 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262923+0000", 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4913999999999994e-05 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263033+0000", 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110017 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:464 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1322409250000001, 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047436980000000002, 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 464 2024-09-18T01:02:49.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258456+0000", 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.78e-07 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258460+0000", 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9330000000000003e-06 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258470+0000", 2024-09-18T01:02:49.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0108e-05 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258476+0000", 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263068+0000", 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045913220000000001 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263137+0000", 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9513999999999999e-05 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263199+0000", 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:775 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.132207159, 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004922844, 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 775 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258490+0000", 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258494+0000", 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258505+0000", 2024-09-18T01:02:49.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1269e-05 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258512+0000", 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.182e-06 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263231+0000", 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047188259999999997 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263299+0000", 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8009999999999994e-05 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263412+0000", 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011239399999999999 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:465 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1318349149999998, 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004784071, 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 465 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258862+0000", 2024-09-18T01:02:49.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5099999999999998e-07 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258867+0000", 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2700000000000004e-06 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258879+0000", 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1457999999999999e-05 2024-09-18T01:02:49.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258886+0000", 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-06 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263461+0000", 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004574771 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263546+0000", 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4816999999999998e-05 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263645+0000", 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9382000000000005e-05 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:776 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1317772079999999, 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004957576, 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 776 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4399999999999999e-07 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258924+0000", 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258933+0000", 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1360000000000003e-06 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258940+0000", 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5830000000000003e-06 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263684+0000", 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047445070000000002 2024-09-18T01:02:49.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263764+0000", 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9754999999999998e-05 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263876+0000", 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112391 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:717 3.b 3:d07cefe4:::rbd_data.1103b05c2cf1.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.9613793560000001, 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046200909999999998, 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 717 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.92e-07 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429321+0000", 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8759999999999998e-06 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429331+0000", 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4700000000000008e-06 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429339+0000", 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.038e-06 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429410+0000", 2024-09-18T01:02:49.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1299999999999998e-05 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429492+0000", 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2108999999999996e-05 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429611+0000", 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011903100000000001 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430998+0000", 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001386649 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433835+0000", 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028371289999999999 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433892+0000", 2024-09-18T01:02:49.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629e-05 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433937+0000", 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4168000000000001e-05 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:722 3.b 3:d7118d93:::rbd_data.1103b05c2cf1.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.9612652690000001, 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046371340000000002, 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 722 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:49.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429434+0000", 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.621e-06 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429440+0000", 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9959999999999999e-06 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429445+0000", 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0420000000000002e-06 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429846+0000", 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040077599999999999 2024-09-18T01:02:49.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429929+0000", 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3208999999999996e-05 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430033+0000", 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104388 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:49.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433745+0000", 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037116940000000002 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433978+0000", 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023300400000000001 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.434026+0000", 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7725999999999997e-05 2024-09-18T01:02:49.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.434068+0000", 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2116000000000001e-05 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:767 3.2 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:49.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.9434105019999999, 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061286680000000003, 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 767 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447286+0000", 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9599999999999999e-07 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447289+0000", 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.853e-06 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447300+0000", 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1005999999999999e-05 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447307+0000", 2024-09-18T01:02:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447346+0000", 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9787999999999999e-05 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447417+0000", 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0681999999999999e-05 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447592+0000", 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017477700000000001 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.449299+0000", 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017074880000000001 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453334+0000", 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040350669999999998 2024-09-18T01:02:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453370+0000", 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5997000000000001e-05 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453414+0000", 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3550999999999997e-05 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4489.0:830 3.1a e27/20 3:5f37e58f:::rbd_data.1103b05c2cf1.0000000000000018:head v 27'10, mlcod=26'8)", 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.489872+0000", 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.900823428, 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042941990000000003, 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489872+0000", 2024-09-18T01:02:49.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489872+0000", 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489873+0000", 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.28e-07 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489886+0000", 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3389e-05 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489895+0000", 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460000000000006e-06 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489903+0000", 2024-09-18T01:02:49.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5920000000000001e-06 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489971+0000", 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8083000000000005e-05 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489999+0000", 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7124e-05 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.494122+0000", 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004123753 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.494167+0000", 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4283999999999998e-05 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:950 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.573935+0000", 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8167605789999999, 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040303429999999996, 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 950 2024-09-18T01:02:49.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.573935+0000", 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.573935+0000", 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.573936+0000", 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8400000000000002e-07 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.573940+0000", 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2189999999999998e-06 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.573949+0000", 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5040000000000002e-06 2024-09-18T01:02:49.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.573958+0000", 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.576698+0000", 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002740414 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for rw locks", 2024-09-18T01:02:49.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.576783+0000", 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5457999999999994e-05 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.577871+0000", 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010880029999999999 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.577909+0000", 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8031999999999998e-05 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.577966+0000", 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6205999999999997e-05 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:915 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.574101+0000", 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.816594657, 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040824169999999996, 2024-09-18T01:02:49.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 915 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574101+0000", 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574101+0000", 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574102+0000", 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.09e-07 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574106+0000", 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574115+0000", 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2e-06 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7980999999999999e-05 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578005+0000", 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038717159999999999 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578083+0000", 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8397000000000002e-05 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578184+0000", 2024-09-18T01:02:49.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100127 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:951 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.816563191, 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041697490000000004, 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 951 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:49.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1399999999999996e-07 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574137+0000", 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.659e-06 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574144+0000", 2024-09-18T01:02:49.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8719999999999996e-06 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574148+0000", 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0679999999999998e-06 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578211+0000", 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040630780000000003 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:49.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578248+0000", 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7438000000000003e-05 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578302+0000", 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3919999999999999e-05 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:49.500 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:49.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:02:49.502 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:49.503 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:49.503 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:49.503 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:49.503 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:49.504 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:49.504 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:49.500+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:49.504 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:49.505 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:49.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:02:49.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:49.523 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:49.540 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:49.602 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:49.601+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:49.619 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:49.619 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:49.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:02:49.684 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-18T01:02:49.703 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:49.702+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:49.739 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:49.740 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:49.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:02:49.804 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:49.803+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:49.899 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:49.899 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:49.905 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:49.904+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:49.908 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:02:49.986 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xdd98200 2024-09-18T01:02:49.993 INFO:teuthology.orchestra.run.smithi164.stdout:342 trunc from 0x79b7a00 to 0xdd98200 2024-09-18T01:02:50.006 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:50.005+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:50.012 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.012 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.015 INFO:teuthology.orchestra.run.smithi164.stdout:345 trunc from 0xdd98200 to 0x8bff600 2024-09-18T01:02:50.021 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:02:50.047 INFO:teuthology.orchestra.run.smithi164.stdout:346 trunc from 0x8bff600 to 0x97e1800 2024-09-18T01:02:50.056 INFO:teuthology.orchestra.run.smithi164.stdout:347 read 0x65de400 thru 0x65eda70 (0xf671 bytes) 2024-09-18T01:02:50.057 INFO:teuthology.orchestra.run.smithi164.stdout:348 punch from 0x8f68a00 to 0x8f6f600, (0x6c00 bytes) 2024-09-18T01:02:50.057 INFO:teuthology.orchestra.run.smithi164.stdout:349 punch from 0x1902c00 to 0x1903a00, (0xe00 bytes) 2024-09-18T01:02:50.057 INFO:teuthology.orchestra.run.smithi164.stdout:350 punch from 0x7732400 to 0x773d400, (0xb000 bytes) 2024-09-18T01:02:50.059 INFO:teuthology.orchestra.run.smithi164.stdout:351 trunc from 0x97e1800 to 0xb201000 2024-09-18T01:02:50.070 INFO:teuthology.orchestra.run.smithi164.stdout:353 punch from 0x5dda400 to 0x5de1200, (0x6e00 bytes) 2024-09-18T01:02:50.076 INFO:teuthology.orchestra.run.smithi164.stdout:354 punch from 0x223ec00 to 0x224d600, (0xea00 bytes) 2024-09-18T01:02:50.076 INFO:teuthology.orchestra.run.smithi164.stdout:356 read 0x2501e00 thru 0x2501e2c (0x2d bytes) 2024-09-18T01:02:50.078 INFO:teuthology.orchestra.run.smithi164.stdout:357 read 0x72dc800 thru 0x72e5676 (0x8e77 bytes) 2024-09-18T01:02:50.079 INFO:teuthology.orchestra.run.smithi164.stdout:358 write 0x1d46200 thru 0x1d53c78 (0xda79 bytes) 2024-09-18T01:02:50.079 INFO:teuthology.orchestra.run.smithi164.stdout:359 read 0x2e4f400 thru 0x2e553e4 (0x5fe5 bytes) 2024-09-18T01:02:50.079 INFO:teuthology.orchestra.run.smithi164.stdout:361 write 0x8443e00 thru 0x8450cdc (0xcedd bytes) 2024-09-18T01:02:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:363 read 0x7c94400 thru 0x7c9b7a0 (0x73a1 bytes) 2024-09-18T01:02:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:364 read 0x5eaa000 thru 0x5eaa52d (0x52e bytes) 2024-09-18T01:02:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:365 read 0x42ddc00 thru 0x42e6684 (0x8a85 bytes) 2024-09-18T01:02:50.081 INFO:teuthology.orchestra.run.smithi164.stdout:366 punch from 0x5eeee00 to 0x5ef0000, (0x1200 bytes) 2024-09-18T01:02:50.081 INFO:teuthology.orchestra.run.smithi164.stdout:367 punch from 0x3010800 to 0x301b800, (0xb000 bytes) 2024-09-18T01:02:50.082 INFO:teuthology.orchestra.run.smithi164.stdout:368 punch from 0x46f2000 to 0x46f5600, (0x3600 bytes) 2024-09-18T01:02:50.082 INFO:teuthology.orchestra.run.smithi164.stdout:369 read 0x86da800 thru 0x86e9a4f (0xf250 bytes) 2024-09-18T01:02:50.092 INFO:teuthology.orchestra.run.smithi164.stdout:371 trunc from 0xb201000 to 0x4b3fe00 2024-09-18T01:02:50.107 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:50.106+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:50.121 INFO:teuthology.orchestra.run.smithi164.stdout:373 read 0x2d6a400 thru 0x2d74193 (0x9d94 bytes) 2024-09-18T01:02:50.132 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.132 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.136 INFO:teuthology.orchestra.run.smithi164.stdout:374 write 0xa14e000 thru 0xa152bf9 (0x4bfa bytes) 2024-09-18T01:02:50.137 INFO:teuthology.orchestra.run.smithi164.stdout:376 punch from 0x76c9400 to 0x76d7c00, (0xe800 bytes) 2024-09-18T01:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout:377 read 0x5f2f400 thru 0x5f344e8 (0x50e9 bytes) 2024-09-18T01:02:50.139 INFO:teuthology.orchestra.run.smithi164.stdout:378 read 0x1a6b200 thru 0x1a7582b (0xa62c bytes) 2024-09-18T01:02:50.142 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:02:50.153 INFO:teuthology.orchestra.run.smithi164.stdout:380 write 0xd88ac00 thru 0xd88bf3c (0x133d bytes) 2024-09-18T01:02:50.154 INFO:teuthology.orchestra.run.smithi164.stdout:382 read 0xafc6000 thru 0xafd54a2 (0xf4a3 bytes) 2024-09-18T01:02:50.161 INFO:teuthology.orchestra.run.smithi164.stdout:384 trunc from 0xd88c000 to 0x8ae9000 2024-09-18T01:02:50.199 INFO:teuthology.orchestra.run.smithi164.stdout:385 write 0xcf3d800 thru 0xcf45dd2 (0x85d3 bytes) 2024-09-18T01:02:50.200 INFO:teuthology.orchestra.run.smithi164.stdout:389 punch from 0xb1e4e00 to 0xb1f0a00, (0xbc00 bytes) 2024-09-18T01:02:50.204 INFO:teuthology.orchestra.run.smithi164.stdout:390 read 0x552fc00 thru 0x5539b30 (0x9f31 bytes) 2024-09-18T01:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout:391 read 0x978ce00 thru 0x9792064 (0x5265 bytes) 2024-09-18T01:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout:393 punch from 0x6b6600 to 0x6c1800, (0xb200 bytes) 2024-09-18T01:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout:394 read 0x2f0a200 thru 0x2f0de37 (0x3c38 bytes) 2024-09-18T01:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout:395 read 0xa1b5a00 thru 0xa1be0d7 (0x86d8 bytes) 2024-09-18T01:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout:396 read 0xc82e400 thru 0xc839bb6 (0xb7b7 bytes) 2024-09-18T01:02:50.205 INFO:teuthology.orchestra.run.smithi164.stdout:399 write 0xc047800 thru 0xc0539fc (0xc1fd bytes) 2024-09-18T01:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout:400 read 0x3da7600 thru 0x3daf41d (0x7e1e bytes) 2024-09-18T01:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout:401 write 0xa9a6800 thru 0xa9b26b1 (0xbeb2 bytes) 2024-09-18T01:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout:402 read 0xbd1fc00 thru 0xbd2d511 (0xd912 bytes) 2024-09-18T01:02:50.206 INFO:teuthology.orchestra.run.smithi164.stdout:403 write 0x2f2d600 thru 0x2f2f336 (0x1d37 bytes) 2024-09-18T01:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout:405 write 0x3d75200 thru 0x3d7dbe5 (0x89e6 bytes) 2024-09-18T01:02:50.207 INFO:teuthology.orchestra.run.smithi164.stdout:406 read 0xb4d5c00 thru 0xb4d64d3 (0x8d4 bytes) 2024-09-18T01:02:50.208 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:50.206+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout:407 write 0xb244800 thru 0xb2503b7 (0xbbb8 bytes) 2024-09-18T01:02:50.208 INFO:teuthology.orchestra.run.smithi164.stdout:408 read 0x1430000 thru 0x143a728 (0xa729 bytes) 2024-09-18T01:02:50.214 INFO:teuthology.orchestra.run.smithi164.stdout:409 trunc from 0xcf45e00 to 0x7aba000 2024-09-18T01:02:50.247 INFO:teuthology.orchestra.run.smithi164.stdout:410 trunc from 0x7aba000 to 0xb39200 2024-09-18T01:02:50.254 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.254 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.264 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:02:50.292 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:02:50.328 INFO:teuthology.orchestra.run.smithi164.stdout:411 write 0x5028200 thru 0x5034b60 (0xc961 bytes) 2024-09-18T01:02:50.329 INFO:teuthology.orchestra.run.smithi164.stdout:415 write 0x221d200 thru 0x2221ddd (0x4bde bytes) 2024-09-18T01:02:50.345 INFO:teuthology.orchestra.run.smithi164.stdout:416 write 0x66e4400 thru 0x66f001f (0xbc20 bytes) 2024-09-18T01:02:50.346 INFO:teuthology.orchestra.run.smithi164.stdout:417 read 0x521e00 thru 0x524e6b (0x306c bytes) 2024-09-18T01:02:50.346 INFO:teuthology.orchestra.run.smithi164.stdout:418 punch from 0x1956200 to 0x195ca00, (0x6800 bytes) 2024-09-18T01:02:50.346 INFO:teuthology.orchestra.run.smithi164.stdout:420 read 0x3430200 thru 0x343cb85 (0xc986 bytes) 2024-09-18T01:02:50.348 INFO:teuthology.orchestra.run.smithi164.stdout:422 trunc from 0x66f0200 to 0x53d8e00 2024-09-18T01:02:50.348 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:50.347+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:50.364 INFO:teuthology.orchestra.run.smithi164.stdout:423 write 0x153e400 thru 0x1542399 (0x3f9a bytes) 2024-09-18T01:02:50.365 INFO:teuthology.orchestra.run.smithi164.stdout:424 read 0x9de400 thru 0x9e5d0f (0x7910 bytes) 2024-09-18T01:02:50.392 INFO:teuthology.orchestra.run.smithi164.stdout:425 write 0xe0f6a00 thru 0xe103591 (0xcb92 bytes) 2024-09-18T01:02:50.392 INFO:teuthology.orchestra.run.smithi164.stdout:426 read 0x53da600 thru 0x53e219c (0x7b9d bytes) 2024-09-18T01:02:50.409 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:50.407+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:50.410 INFO:teuthology.orchestra.run.smithi164.stdout:427 write 0xe2c7600 thru 0xe2d37b2 (0xc1b3 bytes) 2024-09-18T01:02:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.418 INFO:teuthology.orchestra.run.smithi164.stdout:429 trunc from 0xe2d3800 to 0x73f9400 2024-09-18T01:02:50.427 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:02:50.448 INFO:teuthology.orchestra.run.smithi164.stdout:430 write 0x84cc800 thru 0x84d5eb7 (0x96b8 bytes) 2024-09-18T01:02:50.455 INFO:teuthology.orchestra.run.smithi164.stdout:433 trunc from 0x84d6000 to 0x276b000 2024-09-18T01:02:50.482 INFO:teuthology.orchestra.run.smithi164.stdout:434 read 0x6c2200 thru 0x6c6e20 (0x4c21 bytes) 2024-09-18T01:02:50.484 INFO:teuthology.orchestra.run.smithi164.stdout:438 punch from 0x1354400 to 0x1362800, (0xe400 bytes) 2024-09-18T01:02:50.485 INFO:teuthology.orchestra.run.smithi164.stdout:439 read 0xecd200 thru 0xed8d93 (0xbb94 bytes) 2024-09-18T01:02:50.485 INFO:teuthology.orchestra.run.smithi164.stdout:442 read 0x36d400 thru 0x3798cc (0xc4cd bytes) 2024-09-18T01:02:50.503 INFO:teuthology.orchestra.run.smithi164.stdout:444 write 0xa9f2a00 thru 0xa9f9478 (0x6a79 bytes) 2024-09-18T01:02:50.504 INFO:teuthology.orchestra.run.smithi164.stdout:447 read 0x6327a00 thru 0x6334388 (0xc989 bytes) 2024-09-18T01:02:50.511 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:50.510+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:50.522 INFO:teuthology.orchestra.run.smithi164.stdout:448 write 0xdccce00 thru 0xdcd46a4 (0x78a5 bytes) 2024-09-18T01:02:50.523 INFO:teuthology.orchestra.run.smithi164.stdout:449 write 0x5253a00 thru 0x525b592 (0x7b93 bytes) 2024-09-18T01:02:50.523 INFO:teuthology.orchestra.run.smithi164.stdout:451 read 0xb620c00 thru 0xb629a89 (0x8e8a bytes) 2024-09-18T01:02:50.523 INFO:teuthology.orchestra.run.smithi164.stdout:452 write 0x8665400 thru 0x86740cb (0xeccc bytes) 2024-09-18T01:02:50.523 INFO:teuthology.orchestra.run.smithi164.stdout:453 punch from 0x5a83800 to 0x5a89600, (0x5e00 bytes) 2024-09-18T01:02:50.525 INFO:teuthology.orchestra.run.smithi164.stdout:456 read 0x9976600 thru 0x997b3d2 (0x4dd3 bytes) 2024-09-18T01:02:50.525 INFO:teuthology.orchestra.run.smithi164.stdout:457 write 0x1558e00 thru 0x1561d65 (0x8f66 bytes) 2024-09-18T01:02:50.525 INFO:teuthology.orchestra.run.smithi164.stdout:459 punch from 0x1587c00 to 0x1597200, (0xf600 bytes) 2024-09-18T01:02:50.526 INFO:teuthology.orchestra.run.smithi164.stdout:461 write 0x5669200 thru 0x566e7cf (0x55d0 bytes) 2024-09-18T01:02:50.526 INFO:teuthology.orchestra.run.smithi164.stdout:463 write 0xca18e00 thru 0xca19edc (0x10dd bytes) 2024-09-18T01:02:50.526 INFO:teuthology.orchestra.run.smithi164.stdout:464 write 0x81d9c00 thru 0x81da8c3 (0xcc4 bytes) 2024-09-18T01:02:50.528 INFO:teuthology.orchestra.run.smithi164.stdout:465 write 0x693e200 thru 0x694d9d1 (0xf7d2 bytes) 2024-09-18T01:02:50.528 INFO:teuthology.orchestra.run.smithi164.stdout:466 read 0x1ccd400 thru 0x1ccd981 (0x582 bytes) 2024-09-18T01:02:50.529 INFO:teuthology.orchestra.run.smithi164.stdout:467 write 0x9be4600 thru 0x9be964c (0x504d bytes) 2024-09-18T01:02:50.529 INFO:teuthology.orchestra.run.smithi164.stdout:470 write 0x811d400 thru 0x8124fe3 (0x7be4 bytes) 2024-09-18T01:02:50.529 INFO:teuthology.orchestra.run.smithi164.stdout:477 read 0x2da5000 thru 0x2db3729 (0xe72a bytes) 2024-09-18T01:02:50.529 INFO:teuthology.orchestra.run.smithi164.stdout:479 punch from 0x5a99000 to 0x5aa4e00, (0xbe00 bytes) 2024-09-18T01:02:50.530 INFO:teuthology.orchestra.run.smithi164.stdout:481 write 0x4f05800 thru 0x4f10e29 (0xb62a bytes) 2024-09-18T01:02:50.530 INFO:teuthology.orchestra.run.smithi164.stdout:483 punch from 0x5187000 to 0x5193c00, (0xcc00 bytes) 2024-09-18T01:02:50.530 INFO:teuthology.orchestra.run.smithi164.stdout:484 write 0xa455c00 thru 0xa458dfd (0x31fe bytes) 2024-09-18T01:02:50.533 INFO:teuthology.orchestra.run.smithi164.stdout:486 trunc from 0xdcd4800 to 0xb82fc00 2024-09-18T01:02:50.557 INFO:teuthology.orchestra.run.smithi164.stdout:488 write 0x89afc00 thru 0x89b7ebd (0x82be bytes) 2024-09-18T01:02:50.558 INFO:teuthology.orchestra.run.smithi164.stdout:490 read 0x7581800 thru 0x7583fe2 (0x27e3 bytes) 2024-09-18T01:02:50.573 INFO:teuthology.orchestra.run.smithi164.stdout:492 write 0xe7a0600 thru 0xe7af55a (0xef5b bytes) 2024-09-18T01:02:50.574 INFO:teuthology.orchestra.run.smithi164.stdout:497 read 0x789a800 thru 0x78a0902 (0x6103 bytes) 2024-09-18T01:02:50.575 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xecb4400 2024-09-18T01:02:50.575 INFO:teuthology.orchestra.run.smithi164.stdout:499 trunc from 0xe7af600 to 0xecb4400 2024-09-18T01:02:50.580 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.580 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.589 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:02:50.593 INFO:teuthology.orchestra.run.smithi164.stdout:500 read 0x8591600 thru 0x85931fc (0x1bfd bytes) 2024-09-18T01:02:50.593 INFO:teuthology.orchestra.run.smithi164.stdout:501 read 0x6669800 thru 0x666e07d (0x487e bytes) 2024-09-18T01:02:50.596 INFO:teuthology.orchestra.run.smithi164.stdout:502 trunc from 0xecb4400 to 0xd5bca00 2024-09-18T01:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout:503 write 0x68af800 thru 0x68b7e36 (0x8637 bytes) 2024-09-18T01:02:50.619 INFO:teuthology.orchestra.run.smithi164.stdout:504 write 0x83bb400 thru 0x83bf9ef (0x45f0 bytes) 2024-09-18T01:02:50.620 INFO:teuthology.orchestra.run.smithi164.stdout:507 trunc from 0xd5bca00 to 0xd113200 2024-09-18T01:02:50.630 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:50.629+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:50.632 INFO:teuthology.orchestra.run.smithi164.stdout:508 punch from 0xa78200 to 0xa7c600, (0x4400 bytes) 2024-09-18T01:02:50.637 INFO:teuthology.orchestra.run.smithi164.stdout:509 trunc from 0xd113200 to 0xde83600 2024-09-18T01:02:50.644 INFO:teuthology.orchestra.run.smithi164.stdout:513 punch from 0x74a9a00 to 0x74b2c00, (0x9200 bytes) 2024-09-18T01:02:50.647 INFO:teuthology.orchestra.run.smithi164.stdout:516 write 0x312f800 thru 0x313ced5 (0xd6d6 bytes) 2024-09-18T01:02:50.649 INFO:teuthology.orchestra.run.smithi164.stdout:519 read 0x8d0f600 thru 0x8d1b247 (0xbc48 bytes) 2024-09-18T01:02:50.663 INFO:teuthology.orchestra.run.smithi164.stdout:520 trunc from 0xde83600 to 0xfeaa00 2024-09-18T01:02:50.711 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:50.710+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:50.725 INFO:teuthology.orchestra.run.smithi164.stdout:521 punch from 0x8c6600 to 0x8c7000, (0xa00 bytes) 2024-09-18T01:02:50.743 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.743 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.744 INFO:teuthology.orchestra.run.smithi164.stdout:522 write 0x5405200 thru 0x540791d (0x271e bytes) 2024-09-18T01:02:50.753 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:02:50.768 INFO:teuthology.orchestra.run.smithi164.stdout:523 write 0xcc67600 thru 0xcc7085a (0x925b bytes) 2024-09-18T01:02:50.782 INFO:teuthology.orchestra.run.smithi164.stdout:524 trunc from 0xcc70a00 to 0xc0aa00 2024-09-18T01:02:50.812 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:50.811+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:50.814 INFO:teuthology.orchestra.run.smithi164.stdout:528 read 0x1fe000 thru 0x2025c6 (0x45c7 bytes) 2024-09-18T01:02:50.814 INFO:teuthology.orchestra.run.smithi164.stdout:529 read 0x727200 thru 0x728fbf (0x1dc0 bytes) 2024-09-18T01:02:50.839 INFO:teuthology.orchestra.run.smithi164.stdout:531 write 0x8325800 thru 0x832f0c5 (0x98c6 bytes) 2024-09-18T01:02:50.839 INFO:teuthology.orchestra.run.smithi164.stdout:536 write 0xcade00 thru 0xcbb512 (0xd713 bytes) 2024-09-18T01:02:50.839 INFO:teuthology.orchestra.run.smithi164.stdout:540 read 0x2970000 thru 0x297bef0 (0xbef1 bytes) 2024-09-18T01:02:50.848 INFO:teuthology.orchestra.run.smithi164.stdout:541 trunc from 0x832f200 to 0x408400 2024-09-18T01:02:50.864 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.864 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.873 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:02:50.879 INFO:teuthology.orchestra.run.smithi164.stdout:542 read 0x38a600 thru 0x38b21b (0xc1c bytes) 2024-09-18T01:02:50.894 INFO:teuthology.orchestra.run.smithi164.stdout:543 trunc from 0x408400 to 0xc7bd600 2024-09-18T01:02:50.905 INFO:teuthology.orchestra.run.smithi164.stdout:545 write 0x430dc00 thru 0x431c56b (0xe96c bytes) 2024-09-18T01:02:50.912 INFO:teuthology.orchestra.run.smithi164.stdout:548 read 0x28e1c00 thru 0x28eb90c (0x9d0d bytes) 2024-09-18T01:02:50.912 INFO:teuthology.orchestra.run.smithi164.stdout:549 write 0x5d0dc00 thru 0x5d134ea (0x58eb bytes) 2024-09-18T01:02:50.917 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:50.915+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:50.924 INFO:teuthology.orchestra.run.smithi164.stdout:550 trunc from 0xc7bd600 to 0x161e400 2024-09-18T01:02:50.951 INFO:teuthology.orchestra.run.smithi164.stdout:551 read 0x12b1a00 thru 0x12bdf46 (0xc547 bytes) 2024-09-18T01:02:50.952 INFO:teuthology.orchestra.run.smithi164.stdout:552 write 0x80fe00 thru 0x8172a1 (0x74a2 bytes) 2024-09-18T01:02:50.958 INFO:teuthology.orchestra.run.smithi164.stdout:555 trunc from 0x161e400 to 0x65f9400 2024-09-18T01:02:50.969 INFO:teuthology.orchestra.run.smithi164.stdout:556 read 0x3b66000 thru 0x3b7172c (0xb72d bytes) 2024-09-18T01:02:50.970 INFO:teuthology.orchestra.run.smithi164.stdout:559 read 0x1ef8400 thru 0x1f04868 (0xc469 bytes) 2024-09-18T01:02:50.972 INFO:teuthology.orchestra.run.smithi164.stdout:560 trunc from 0x65f9400 to 0x4bb9400 2024-09-18T01:02:50.981 INFO:teuthology.orchestra.run.smithi164.stdout:561 read 0x4216800 thru 0x4223ad9 (0xd2da bytes) 2024-09-18T01:02:50.981 INFO:teuthology.orchestra.run.smithi164.stdout:562 punch from 0xf22000 to 0xf2c200, (0xa200 bytes) 2024-09-18T01:02:50.983 INFO:teuthology.orchestra.run.smithi164.stdout:563 read 0x4090800 thru 0x409a9a6 (0xa1a7 bytes) 2024-09-18T01:02:50.984 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:50.984 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:50.992 INFO:teuthology.orchestra.run.smithi164.stdout:564 trunc from 0x4bb9400 to 0xca0ae00 2024-09-18T01:02:50.993 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:02:51.002 INFO:teuthology.orchestra.run.smithi164.stdout:565 write 0x1546600 thru 0x154e9ff (0x8400 bytes) 2024-09-18T01:02:51.002 INFO:teuthology.orchestra.run.smithi164.stdout:567 punch from 0xb352e00 to 0xb362200, (0xf400 bytes) 2024-09-18T01:02:51.005 INFO:teuthology.orchestra.run.smithi164.stdout:568 write 0x47d8e00 thru 0x47da7a3 (0x19a4 bytes) 2024-09-18T01:02:51.005 INFO:teuthology.orchestra.run.smithi164.stdout:570 write 0x9b48600 thru 0x9b517c7 (0x91c8 bytes) 2024-09-18T01:02:51.005 INFO:teuthology.orchestra.run.smithi164.stdout:571 punch from 0x54a2600 to 0x54afc00, (0xd600 bytes) 2024-09-18T01:02:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:572 punch from 0xbbeec00 to 0xbbf6e00, (0x8200 bytes) 2024-09-18T01:02:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:573 write 0x2593000 thru 0x259b39b (0x839c bytes) 2024-09-18T01:02:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:574 read 0x851f600 thru 0x8522837 (0x3238 bytes) 2024-09-18T01:02:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:575 write 0x2d83400 thru 0x2d91b6e (0xe76f bytes) 2024-09-18T01:02:51.006 INFO:teuthology.orchestra.run.smithi164.stdout:578 read 0x5e64e00 thru 0x5e7160c (0xc80d bytes) 2024-09-18T01:02:51.016 INFO:teuthology.orchestra.run.smithi164.stdout:580 trunc from 0xca0ae00 to 0x3c63a00 2024-09-18T01:02:51.034 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:51.032+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:51.051 INFO:teuthology.orchestra.run.smithi164.stdout:583 read 0x1abc600 thru 0x1abd2a7 (0xca8 bytes) 2024-09-18T01:02:51.051 INFO:teuthology.orchestra.run.smithi164.stdout:585 punch from 0x1b85a00 to 0x1b8fc00, (0xa200 bytes) 2024-09-18T01:02:51.062 INFO:teuthology.orchestra.run.smithi164.stdout:591 trunc from 0x3c63a00 to 0xb246e00 2024-09-18T01:02:51.071 INFO:teuthology.orchestra.run.smithi164.stdout:593 write 0x39000 thru 0x3d27c (0x427d bytes) 2024-09-18T01:02:51.073 INFO:teuthology.orchestra.run.smithi164.stdout:594 punch from 0x26a8000 to 0x26a8c00, (0xc00 bytes) 2024-09-18T01:02:51.086 INFO:teuthology.orchestra.run.smithi164.stdout:596 write 0xe188c00 thru 0xe18a63a (0x1a3b bytes) 2024-09-18T01:02:51.094 INFO:teuthology.orchestra.run.smithi164.stdout:597 read 0x4a4c600 thru 0x4a5658a (0x9f8b bytes) 2024-09-18T01:02:51.102 INFO:teuthology.orchestra.run.smithi164.stdout:598 trunc from 0xe18a800 to 0x7146a00 2024-09-18T01:02:51.114 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:51.113+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:51.143 INFO:teuthology.orchestra.run.smithi164.stdout:599 write 0xc29da00 thru 0xc2a12e0 (0x38e1 bytes) 2024-09-18T01:02:51.144 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:51.145 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:51.148 INFO:teuthology.orchestra.run.smithi164.stdout:600 punch from 0x727c600 to 0x727cc00, (0x600 bytes) 2024-09-18T01:02:51.150 INFO:teuthology.orchestra.run.smithi164.stdout:602 write 0xa24bc00 thru 0xa255bdf (0x9fe0 bytes) 2024-09-18T01:02:51.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:51.166 INFO:teuthology.orchestra.run.smithi164.stdout:603 write 0xd289800 thru 0xd296b51 (0xd352 bytes) 2024-09-18T01:02:51.173 INFO:teuthology.orchestra.run.smithi164.stdout:604 read 0x4756a00 thru 0x4756e5c (0x45d bytes) 2024-09-18T01:02:51.190 INFO:teuthology.orchestra.run.smithi164.stdout:605 write 0xed1f800 thru 0xed26d19 (0x751a bytes) 2024-09-18T01:02:51.194 INFO:teuthology.orchestra.run.smithi164.stdout:606 write 0xdc51c00 thru 0xdc55d23 (0x4124 bytes) 2024-09-18T01:02:51.195 INFO:teuthology.orchestra.run.smithi164.stdout:608 read 0x8d5c00 thru 0x8d9869 (0x3c6a bytes) 2024-09-18T01:02:51.195 INFO:teuthology.orchestra.run.smithi164.stdout:610 write 0xdfe6a00 thru 0xdfe6d18 (0x319 bytes) 2024-09-18T01:02:51.195 INFO:teuthology.orchestra.run.smithi164.stdout:612 write 0x311f200 thru 0x312c720 (0xd521 bytes) 2024-09-18T01:02:51.205 INFO:teuthology.orchestra.run.smithi164.stdout:615 trunc from 0xed26e00 to 0x8018a00 2024-09-18T01:02:51.215 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:51.213+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:51.231 INFO:teuthology.orchestra.run.smithi164.stdout:618 punch from 0x1878000 to 0x187a400, (0x2400 bytes) 2024-09-18T01:02:51.257 INFO:teuthology.orchestra.run.smithi164.stdout:620 write 0xe8e7200 thru 0xe8f5008 (0xde09 bytes) 2024-09-18T01:02:51.263 INFO:teuthology.orchestra.run.smithi164.stdout:621 write 0xbdf1c00 thru 0xbdfa388 (0x8789 bytes) 2024-09-18T01:02:51.268 INFO:teuthology.orchestra.run.smithi164.stdout:622 trunc from 0xe8f5200 to 0xab48a00 2024-09-18T01:02:51.269 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:51.269 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:51.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:51.291 INFO:teuthology.orchestra.run.smithi164.stdout:623 read 0xf09200 thru 0xf0b4b8 (0x22b9 bytes) 2024-09-18T01:02:51.308 INFO:teuthology.orchestra.run.smithi164.stdout:627 write 0xbd9e800 thru 0xbda9378 (0xab79 bytes) 2024-09-18T01:02:51.313 INFO:teuthology.orchestra.run.smithi164.stdout:628 read 0x96de000 thru 0x96ed9a9 (0xf9aa bytes) 2024-09-18T01:02:51.313 INFO:teuthology.orchestra.run.smithi164.stdout:629 read 0x5152e00 thru 0x5159911 (0x6b12 bytes) 2024-09-18T01:02:51.314 INFO:teuthology.orchestra.run.smithi164.stdout:631 punch from 0x3971a00 to 0x397c800, (0xae00 bytes) 2024-09-18T01:02:51.314 INFO:teuthology.orchestra.run.smithi164.stdout:632 write 0x8495400 thru 0x849c9e6 (0x75e7 bytes) 2024-09-18T01:02:51.314 INFO:teuthology.orchestra.run.smithi164.stdout:633 write 0xa2f4200 thru 0xa2f8a4b (0x484c bytes) 2024-09-18T01:02:51.315 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:51.313+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:51.316 INFO:teuthology.orchestra.run.smithi164.stdout:636 punch from 0xb81ca00 to 0xb827e00, (0xb400 bytes) 2024-09-18T01:02:51.316 INFO:teuthology.orchestra.run.smithi164.stdout:638 read 0x7cc9a00 thru 0x7cd417f (0xa780 bytes) 2024-09-18T01:02:51.317 INFO:teuthology.orchestra.run.smithi164.stdout:639 read 0xb1c1e00 thru 0xb1c8c3c (0x6e3d bytes) 2024-09-18T01:02:51.325 INFO:teuthology.orchestra.run.smithi164.stdout:640 trunc from 0xbda9400 to 0x55a1600 2024-09-18T01:02:51.362 INFO:teuthology.orchestra.run.smithi164.stdout:641 write 0xb686a00 thru 0xb68d489 (0x6a8a bytes) 2024-09-18T01:02:51.367 INFO:teuthology.orchestra.run.smithi164.stdout:642 read 0x6fb1400 thru 0x6fb5d3f (0x4940 bytes) 2024-09-18T01:02:51.369 INFO:teuthology.orchestra.run.smithi164.stdout:643 punch from 0x82a5e00 to 0x82b4a00, (0xec00 bytes) 2024-09-18T01:02:51.369 INFO:teuthology.orchestra.run.smithi164.stdout:646 read 0x380000 thru 0x383304 (0x3305 bytes) 2024-09-18T01:02:51.382 INFO:teuthology.orchestra.run.smithi164.stdout:647 trunc from 0xb68d600 to 0xc72200 2024-09-18T01:02:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:51.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:51.452 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:51.451+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:51.459 INFO:teuthology.orchestra.run.smithi164.stdout:651 trunc from 0xc72200 to 0xd940c00 2024-09-18T01:02:51.475 INFO:teuthology.orchestra.run.smithi164.stdout:653 write 0x50b3c00 thru 0x50b581a (0x1c1b bytes) 2024-09-18T01:02:51.489 INFO:teuthology.orchestra.run.smithi164.stdout:654 write 0xed4aa00 thru 0xed4af20 (0x521 bytes) 2024-09-18T01:02:51.491 INFO:teuthology.orchestra.run.smithi164.stdout:655 trunc from 0xed4b000 to 0xde1de00 2024-09-18T01:02:51.507 INFO:teuthology.orchestra.run.smithi164.stdout:656 read 0x9fb9600 thru 0x9fc5322 (0xbd23 bytes) 2024-09-18T01:02:51.508 INFO:teuthology.orchestra.run.smithi164.stdout:657 write 0x8cc5c00 thru 0x8ccdfb8 (0x83b9 bytes) 2024-09-18T01:02:51.517 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:51.517 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:51.518 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:51.516+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:51.518 INFO:teuthology.orchestra.run.smithi164.stdout:659 trunc from 0xde1de00 to 0x5c83c00 2024-09-18T01:02:51.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:02:51.533 INFO:teuthology.orchestra.run.smithi164.stdout:664 punch from 0x35b1800 to 0x35bca00, (0xb200 bytes) 2024-09-18T01:02:51.551 INFO:teuthology.orchestra.run.smithi164.stdout:665 write 0xb12e200 thru 0xb13d917 (0xf718 bytes) 2024-09-18T01:02:51.552 INFO:teuthology.orchestra.run.smithi164.stdout:668 clone 3 order 20 su 65536 sc 2 2024-09-18T01:02:51.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:02:51.618 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:51.617+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:51.677 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:51.678 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:51.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:02:51.691 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:51.691 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:51.709 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:51.719 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:51.718+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:51.725 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:51.743 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:51.743 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:51.759 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:51.777 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:51.777 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:51.795 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:51.795 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:51.795 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:51.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:02:51.820 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:51.819+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:51.921 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:51.919+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:51.952 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:771 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.257727+0000", 2024-09-18T01:02:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6925248079999999, 2024-09-18T01:02:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041119729999999997, 2024-09-18T01:02:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 771 2024-09-18T01:02:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257727+0000", 2024-09-18T01:02:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257727+0000", 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257728+0000", 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5899999999999995e-07 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257732+0000", 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.78e-06 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257744+0000", 2024-09-18T01:02:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1259e-05 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257752+0000", 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1100000000000003e-06 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261707+0000", 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039551129999999997 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261770+0000", 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3398000000000005e-05 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261839+0000", 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8454000000000006e-05 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:460 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.257937+0000", 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6923148550000002, 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042138870000000004, 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 460 2024-09-18T01:02:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257937+0000", 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257937+0000", 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257938+0000", 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6299999999999993e-07 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257942+0000", 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257954+0000", 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1883e-05 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257970+0000", 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6277999999999999e-05 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262010+0000", 2024-09-18T01:02:51.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040396169999999997 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262046+0000", 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6381000000000003e-05 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262151+0000", 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010419799999999999 2024-09-18T01:02:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:773 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258077+0000", 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6921745020000003, 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042906730000000001, 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 773 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258077+0000", 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258077+0000", 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258078+0000", 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1399999999999996e-07 2024-09-18T01:02:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258082+0000", 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4830000000000001e-06 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258093+0000", 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0861e-05 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258100+0000", 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6800000000000004e-06 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262199+0000", 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040993239999999997 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262280+0000", 2024-09-18T01:02:51.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1094999999999995e-05 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262368+0000", 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7515999999999994e-05 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:461 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258099+0000", 2024-09-18T01:02:51.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6921529680000003, 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044326670000000004, 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 461 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258099+0000", 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258099+0000", 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258100+0000", 2024-09-18T01:02:51.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.071e-06 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258104+0000", 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3939999999999998e-06 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258114+0000", 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0225e-05 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258129+0000", 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4941e-05 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262396+0000", 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042670670000000003 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262433+0000", 2024-09-18T01:02:51.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6616000000000003e-05 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262531+0000", 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8352999999999998e-05 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:463 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:51.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6920144349999999, 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045481109999999996, 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 463 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258238+0000", 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4900000000000005e-07 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258242+0000", 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.138e-06 2024-09-18T01:02:51.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258252+0000", 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.005e-05 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258261+0000", 2024-09-18T01:02:51.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8279999999999992e-06 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262585+0000", 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043244470000000004 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262670+0000", 2024-09-18T01:02:51.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4451999999999997e-05 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262785+0000", 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011534700000000001 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:774 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6920037820000005, 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047854200000000003, 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 774 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258249+0000", 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7700000000000003e-07 2024-09-18T01:02:51.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258252+0000", 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999998e-06 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258260+0000", 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6750000000000002e-06 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258264+0000", 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2150000000000003e-06 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262838+0000", 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045744319999999998 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262923+0000", 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4913999999999994e-05 2024-09-18T01:02:51.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263033+0000", 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110017 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:464 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6917965690000001, 2024-09-18T01:02:51.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047436980000000002, 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 464 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:51.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258456+0000", 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.78e-07 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258460+0000", 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9330000000000003e-06 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258470+0000", 2024-09-18T01:02:51.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0108e-05 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258476+0000", 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263068+0000", 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045913220000000001 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263137+0000", 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9513999999999999e-05 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263199+0000", 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:775 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6917628029999996, 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004922844, 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 775 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258490+0000", 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:51.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258494+0000", 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258505+0000", 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1269e-05 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258512+0000", 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.182e-06 2024-09-18T01:02:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263231+0000", 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047188259999999997 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263299+0000", 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8009999999999994e-05 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263412+0000", 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011239399999999999 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:465 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:51.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6913905590000002, 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004784071, 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 465 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:51.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258862+0000", 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5099999999999998e-07 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258867+0000", 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2700000000000004e-06 2024-09-18T01:02:52.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258879+0000", 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1457999999999999e-05 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258886+0000", 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-06 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263461+0000", 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004574771 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263546+0000", 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4816999999999998e-05 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263645+0000", 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9382000000000005e-05 2024-09-18T01:02:52.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:776 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6913328520000004, 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004957576, 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 776 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:52.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4399999999999999e-07 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258924+0000", 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258933+0000", 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1360000000000003e-06 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258940+0000", 2024-09-18T01:02:52.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5830000000000003e-06 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263684+0000", 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047445070000000002 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263764+0000", 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9754999999999998e-05 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263876+0000", 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112391 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:717 3.b 3:d07cefe4:::rbd_data.1103b05c2cf1.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5209349999999997, 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046200909999999998, 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 717 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:52.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.92e-07 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429321+0000", 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8759999999999998e-06 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429331+0000", 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4700000000000008e-06 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429339+0000", 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.038e-06 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429410+0000", 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1299999999999998e-05 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429492+0000", 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2108999999999996e-05 2024-09-18T01:02:52.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429611+0000", 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011903100000000001 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430998+0000", 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001386649 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433835+0000", 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028371289999999999 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433892+0000", 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629e-05 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433937+0000", 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4168000000000001e-05 2024-09-18T01:02:52.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:722 3.b 3:d7118d93:::rbd_data.1103b05c2cf1.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5208209129999997, 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046371340000000002, 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 722 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429434+0000", 2024-09-18T01:02:52.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.621e-06 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429440+0000", 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9959999999999999e-06 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429445+0000", 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0420000000000002e-06 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429846+0000", 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040077599999999999 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429929+0000", 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3208999999999996e-05 2024-09-18T01:02:52.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430033+0000", 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104388 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433745+0000", 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037116940000000002 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433978+0000", 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023300400000000001 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.434026+0000", 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7725999999999997e-05 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.434068+0000", 2024-09-18T01:02:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2116000000000001e-05 2024-09-18T01:02:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:767 3.2 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5029661460000003, 2024-09-18T01:02:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061286680000000003, 2024-09-18T01:02:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 767 2024-09-18T01:02:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447286+0000", 2024-09-18T01:02:52.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9599999999999999e-07 2024-09-18T01:02:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447289+0000", 2024-09-18T01:02:52.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.853e-06 2024-09-18T01:02:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447300+0000", 2024-09-18T01:02:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1005999999999999e-05 2024-09-18T01:02:52.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447307+0000", 2024-09-18T01:02:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-18T01:02:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447346+0000", 2024-09-18T01:02:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9787999999999999e-05 2024-09-18T01:02:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447417+0000", 2024-09-18T01:02:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0681999999999999e-05 2024-09-18T01:02:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-18T01:02:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447592+0000", 2024-09-18T01:02:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017477700000000001 2024-09-18T01:02:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.449299+0000", 2024-09-18T01:02:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017074880000000001 2024-09-18T01:02:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453334+0000", 2024-09-18T01:02:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040350669999999998 2024-09-18T01:02:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453370+0000", 2024-09-18T01:02:52.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5997000000000001e-05 2024-09-18T01:02:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453414+0000", 2024-09-18T01:02:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3550999999999997e-05 2024-09-18T01:02:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4489.0:830 3.1a e27/20 3:5f37e58f:::rbd_data.1103b05c2cf1.0000000000000018:head v 27'10, mlcod=26'8)", 2024-09-18T01:02:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.489872+0000", 2024-09-18T01:02:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4603790720000003, 2024-09-18T01:02:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042941990000000003, 2024-09-18T01:02:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489872+0000", 2024-09-18T01:02:52.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489872+0000", 2024-09-18T01:02:52.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489873+0000", 2024-09-18T01:02:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.28e-07 2024-09-18T01:02:52.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489886+0000", 2024-09-18T01:02:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3389e-05 2024-09-18T01:02:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489895+0000", 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460000000000006e-06 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489903+0000", 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5920000000000001e-06 2024-09-18T01:02:52.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489971+0000", 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8083000000000005e-05 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.489999+0000", 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7124e-05 2024-09-18T01:02:52.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.494122+0000", 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004123753 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.494167+0000", 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4283999999999998e-05 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:915 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.574101+0000", 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.376150301, 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040824169999999996, 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 915 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574101+0000", 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574101+0000", 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574102+0000", 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.09e-07 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574106+0000", 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1869999999999999e-06 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574115+0000", 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2e-06 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7980999999999999e-05 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578005+0000", 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038717159999999999 2024-09-18T01:02:52.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578083+0000", 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8397000000000002e-05 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578184+0000", 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100127 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:951 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3761188349999998, 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041697490000000004, 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 951 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:52.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574133+0000", 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1399999999999996e-07 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574137+0000", 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.659e-06 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574144+0000", 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8719999999999996e-06 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.574148+0000", 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0679999999999998e-06 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578211+0000", 2024-09-18T01:02:52.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040630780000000003 2024-09-18T01:02:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578248+0000", 2024-09-18T01:02:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7438000000000003e-05 2024-09-18T01:02:52.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.578302+0000", 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3919999999999999e-05 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:52.057 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6731281770000002, 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:02:52.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:02:52.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:02:52.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:02:52.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6685022739999997, 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:52.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:02:52.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:02:52.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:02:52.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:02:52.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6684242249999999, 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:02:52.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4489.0:1143 3.16 e29/28 3:6e7c3294:::rbd_object_map.1103b05c2cf1.0000000000000004:head v 29'5, mlcod=0'0)", 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:49.301747+0000", 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.6485044049999997, 2024-09-18T01:02:52.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047545599999999997, 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301747+0000", 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301747+0000", 2024-09-18T01:02:52.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301748+0000", 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3200000000000005e-07 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301753+0000", 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.767e-06 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301762+0000", 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6430000000000001e-06 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301769+0000", 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301829+0000", 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0544000000000002e-05 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301853+0000", 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3876e-05 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.306458+0000", 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046053070000000003 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.306502+0000", 2024-09-18T01:02:52.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3284000000000001e-05 2024-09-18T01:02:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.088 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:52.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:02:52.090 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:52.088+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:52.191 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:52.189+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:52.237 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:52.237 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:52.237 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:52.237 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:52.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:02:52.291 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:52.290+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:52.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:02:52.359 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:52.359 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:52.360 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:52.360 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:52.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:02:52.381 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone2 from 0xb13da00 (overlap 0x408400) to 0x135a00 2024-09-18T01:02:52.414 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:52.413+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4208.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.106335214000001, 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00414577, 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378017+0000", 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6500000000000001e-07 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378024+0000", 2024-09-18T01:02:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7300000000000005e-06 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378036+0000", 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1809e-05 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378045+0000", 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2770000000000005e-06 2024-09-18T01:02:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378085+0000", 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0194000000000002e-05 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.378105+0000", 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0693999999999999e-05 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.382107+0000", 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040012729999999996 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:32.382162+0000", 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5627999999999997e-05 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:423 3.18 3:1e8182c7:::rbd_data.1103b05c2cf1.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.253530+0000", 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2308214299999998, 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001608616, 2024-09-18T01:02:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 423 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253530+0000", 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253530+0000", 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253532+0000", 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.519e-06 2024-09-18T01:02:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253538+0000", 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7119999999999997e-06 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253550+0000", 2024-09-18T01:02:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2330000000000001e-05 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253557+0000", 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7710000000000001e-06 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253632+0000", 2024-09-18T01:02:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5383999999999995e-05 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253711+0000", 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8621999999999994e-05 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253823+0000", 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011262199999999999 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.254351+0000", 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052781200000000001 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255031+0000", 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068000700000000005 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255088+0000", 2024-09-18T01:02:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6718999999999997e-05 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255139+0000", 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1118000000000001e-05 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:424 3.1e 3:7947e740:::rbd_data.1103b05c2cf1.0000000000000030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.253586+0000", 2024-09-18T01:02:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2307658730000002, 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024471380000000002, 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 424 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253586+0000", 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253586+0000", 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253586+0000", 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253590+0000", 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.118e-06 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253607+0000", 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7535999999999999e-05 2024-09-18T01:02:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253614+0000", 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4729999999999997e-06 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253670+0000", 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6855999999999999e-05 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253751+0000", 2024-09-18T01:02:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0051999999999999e-05 2024-09-18T01:02:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.253875+0000", 2024-09-18T01:02:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012403499999999999 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.254665+0000", 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079004800000000001 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255945+0000", 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001280663 2024-09-18T01:02:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255993+0000", 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7923000000000002e-05 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256033+0000", 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9878e-05 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:440 3.1e 3:7e1f882a:::rbd_data.1103b05c2cf1.00000000000000f7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.255394+0000", 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2289574649999997, 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017692140000000001, 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 440 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255394+0000", 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255394+0000", 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255395+0000", 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6799999999999999e-07 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255401+0000", 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4369999999999998e-06 2024-09-18T01:02:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255415+0000", 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3981999999999999e-05 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255426+0000", 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1153e-05 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255493+0000", 2024-09-18T01:02:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7037000000000002e-05 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255571+0000", 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8251999999999995e-05 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.255682+0000", 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111137 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256744+0000", 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010622439999999999 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257078+0000", 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033318600000000002 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257119+0000", 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1016999999999997e-05 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257164+0000", 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901000000000003e-05 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4489.0:443 3.0 e26/20 3:0701adeb:::rbd_data.1103b05c2cf1.000000000000010c:head v 26'5, mlcod=26'3)", 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.256282+0000", 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2280697089999997, 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001365479, 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256282+0000", 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256282+0000", 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256283+0000", 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3200000000000005e-07 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256288+0000", 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.874e-06 2024-09-18T01:02:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256298+0000", 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7650000000000005e-06 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256304+0000", 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.998e-06 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256356+0000", 2024-09-18T01:02:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2108999999999999e-05 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.256381+0000", 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4518000000000001e-05 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257599+0000", 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012184470000000001 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.257648+0000", 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8235999999999997e-05 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:469 3.7 3:e4913774:::rbd_data.1103b05c2cf1.000000000000001e:head [stat,truncate 131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.259676+0000", 2024-09-18T01:02:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2246753339999996, 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020690800000000001, 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 469 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259676+0000", 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259676+0000", 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259678+0000", 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.125e-06 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259683+0000", 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259698+0000", 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5159e-05 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259706+0000", 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6559999999999992e-06 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259766+0000", 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0384999999999999e-05 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259862+0000", 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5811999999999997e-05 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.259995+0000", 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013240100000000001 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.260871+0000", 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087623900000000001 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261651+0000", 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078010899999999997 2024-09-18T01:02:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261704+0000", 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2886999999999997e-05 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.261746+0000", 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1786999999999998e-05 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:517 3.1e 3:7e81bc3d:::rbd_data.1103b05c2cf1.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 131072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.270589+0000", 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2137625359999999, 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052699959999999999, 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 517 2024-09-18T01:02:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270589+0000", 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270589+0000", 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270591+0000", 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.939e-06 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270601+0000", 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4429999999999992e-06 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270621+0000", 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0083000000000001e-05 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270631+0000", 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0457e-05 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270705+0000", 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3559000000000005e-05 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.270867+0000", 2024-09-18T01:02:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016231500000000001 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.271080+0000", 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021335700000000001 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.272316+0000", 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012356800000000001 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.275785+0000", 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034685089999999998 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.275818+0000", 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3899000000000002e-05 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.275859+0000", 2024-09-18T01:02:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0754999999999999e-05 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:711 3.7 3:e4913774:::rbd_data.1103b05c2cf1.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.428886+0000", 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0554661950000002, 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002382524, 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 711 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.428886+0000", 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.428886+0000", 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.428887+0000", 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.362e-06 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.428897+0000", 2024-09-18T01:02:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0326000000000001e-05 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.428912+0000", 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4888999999999999e-05 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.428921+0000", 2024-09-18T01:02:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3190000000000001e-06 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429003+0000", 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1062000000000004e-05 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429093+0000", 2024-09-18T01:02:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.064e-05 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429196+0000", 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010307 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429897+0000", 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070064300000000003 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.431179+0000", 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001282187 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.431226+0000", 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7243999999999997e-05 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.431268+0000", 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1782000000000001e-05 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:728 3.1e 3:7e81bc3d:::rbd_data.1103b05c2cf1.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.430204+0000", 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0541482049999997, 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014390640000000001, 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 728 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430204+0000", 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430204+0000", 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430204+0000", 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6899999999999996e-07 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430209+0000", 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5129999999999998e-06 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430224+0000", 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5265e-05 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430234+0000", 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9350000000000006e-06 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430297+0000", 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2663000000000006e-05 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430393+0000", 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6329999999999994e-05 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430521+0000", 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012771599999999999 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.431075+0000", 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00055400700000000002 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.431540+0000", 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046485 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.431598+0000", 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8118999999999997e-05 2024-09-18T01:02:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.431643+0000", 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4796999999999998e-05 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:759 3.18 3:18ce61da:::rbd_data.1103b05c2cf1.0000000000000019:head [stat,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.439092+0000", 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0452593019999998, 2024-09-18T01:02:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041629320000000003, 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 759 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439092+0000", 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439092+0000", 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439094+0000", 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4020000000000001e-06 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439101+0000", 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3610000000000003e-06 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439122+0000", 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0562000000000001e-05 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439130+0000", 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2020000000000004e-06 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439194+0000", 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4262000000000002e-05 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439285+0000", 2024-09-18T01:02:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0252000000000003e-05 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.439456+0000", 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017169 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440079+0000", 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00062245099999999999 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443144+0000", 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030651910000000001 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443203+0000", 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8862e-05 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443255+0000", 2024-09-18T01:02:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2697e-05 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4489.0:764 3.4 e26/20 3:225e67bb:::rbd_data.1103b05c2cf1.000000000000000e:head v 26'13, mlcod=26'11)", 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.441467+0000", 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0428845740000003, 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015592239999999999, 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441467+0000", 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441467+0000", 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441468+0000", 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4799999999999997e-07 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441474+0000", 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3230000000000003e-06 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441484+0000", 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9569999999999997e-06 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441491+0000", 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4350000000000002e-06 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441555+0000", 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4537999999999998e-05 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.441575+0000", 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9238000000000001e-05 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.442984+0000", 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014096460000000001 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.443026+0000", 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2239e-05 2024-09-18T01:02:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:831 3.18 3:18ce61da:::rbd_data.1103b05c2cf1.0000000000000019:head [stat,write 393216~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.487246+0000", 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9971053699999999, 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084221350000000007, 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 831 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487246+0000", 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487246+0000", 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487248+0000", 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9240000000000001e-06 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487281+0000", 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2489e-05 2024-09-18T01:02:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487299+0000", 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7719000000000001e-05 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487306+0000", 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487375+0000", 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8366000000000002e-05 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487384+0000", 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7050000000000005e-06 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493058+0000", 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005674775 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493134+0000", 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5598000000000004e-05 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493292+0000", 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015767600000000001 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495148+0000", 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018560199999999999 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495553+0000", 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040504400000000002 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495619+0000", 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6822999999999994e-05 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495669+0000", 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9066000000000001e-05 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:852 3.1e 3:793934f1:::rbd_data.1103b05c2cf1.000000000000006e:head [set-alloc-hint object_size 524288 write_size 524288,write 305152~22528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.516366+0000", 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9679861150000004, 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020751519999999998, 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 852 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516366+0000", 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516366+0000", 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516367+0000", 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516386+0000", 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8717000000000001e-05 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516400+0000", 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.418e-05 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516407+0000", 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3940000000000001e-06 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516464+0000", 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6969999999999998e-05 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516566+0000", 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102224 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.516655+0000", 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9448000000000001e-05 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.517841+0000", 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011854280000000001 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.518331+0000", 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049016700000000001 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.518393+0000", 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1879999999999997e-05 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.518441+0000", 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8149999999999998e-05 2024-09-18T01:02:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:875 3.18 3:1f8b8eae:::rbd_data.1103b05c2cf1.00000000000000d9:head [set-alloc-hint object_size 524288 write_size 524288,write 423936~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.546114+0000", 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9382375180000002, 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021140540000000002, 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 875 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546114+0000", 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546114+0000", 2024-09-18T01:02:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546116+0000", 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3090000000000001e-06 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546135+0000", 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9051000000000001e-05 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546148+0000", 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3321e-05 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546154+0000", 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9290000000000003e-06 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546212+0000", 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7979000000000002e-05 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546301+0000", 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8666999999999997e-05 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.546381+0000", 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0908999999999994e-05 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.547070+0000", 2024-09-18T01:02:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068810099999999999 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.548169+0000", 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001099076 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.548200+0000", 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0976000000000003e-05 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.548228+0000", 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8736000000000001e-05 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:959 3.18 3:1f8b8eae:::rbd_data.1103b05c2cf1.00000000000000d9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.620790+0000", 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8635618100000002, 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00387726, 2024-09-18T01:02:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 959 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620790+0000", 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620790+0000", 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620791+0000", 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.378e-06 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620797+0000", 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6890000000000001e-06 2024-09-18T01:02:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620813+0000", 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5866e-05 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620828+0000", 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4815e-05 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620901+0000", 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3543999999999999e-05 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.620994+0000", 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2306999999999996e-05 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.621112+0000", 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011838899999999999 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.621707+0000", 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059490699999999999 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.624605+0000", 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028978620000000002 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.624639+0000", 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3958000000000003e-05 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.624667+0000", 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8544999999999999e-05 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1007 3.1e 3:793934f1:::rbd_data.1103b05c2cf1.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.668641+0000", 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8157107660000005, 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001169336, 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1007 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668641+0000", 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668641+0000", 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668642+0000", 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4099999999999997e-07 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668646+0000", 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4939999999999997e-06 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668661+0000", 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5065999999999999e-05 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668670+0000", 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1580000000000005e-06 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668735+0000", 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5040000000000001e-05 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668830+0000", 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5336999999999999e-05 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.668946+0000", 2024-09-18T01:02:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011649 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.669477+0000", 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053062200000000002 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.669738+0000", 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026052699999999998 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.669767+0000", 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9371999999999999e-05 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.669810+0000", 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3389000000000001e-05 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1089 3.7 3:e638ac10:::rbd_data.1103b05c2cf1.000000000000003c:head [set-alloc-hint object_size 524288 write_size 524288,write 196608~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.734404+0000", 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7499474880000001, 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001366033, 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1089 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734404+0000", 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734404+0000", 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734406+0000", 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.296e-06 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734429+0000", 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3686e-05 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734443+0000", 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3437e-05 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734451+0000", 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734505+0000", 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4639000000000003e-05 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734605+0000", 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9522e-05 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.734696+0000", 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1582999999999994e-05 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.735408+0000", 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071190999999999995 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.735708+0000", 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029941499999999998 2024-09-18T01:02:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.735740+0000", 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2570999999999997e-05 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.735770+0000", 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0171e-05 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1133 3.7 3:e598135f:::rbd_data.1103b05c2cf1.00000000000000f5:head [set-alloc-hint object_size 524288 write_size 524288,write 179200~48640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.766440+0000", 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7179116719999996, 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001197827, 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1133 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766440+0000", 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766440+0000", 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766446+0000", 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4550000000000003e-06 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766481+0000", 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5782999999999999e-05 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766493+0000", 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1678999999999999e-05 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766500+0000", 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3769999999999998e-06 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766551+0000", 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0249e-05 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766633+0000", 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2223000000000002e-05 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.766723+0000", 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0069999999999997e-05 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.767386+0000", 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00066348700000000004 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.767578+0000", 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191931 2024-09-18T01:02:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.767611+0000", 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2478999999999999e-05 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.767638+0000", 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7093999999999998e-05 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1934 3.5 3:a7daa15d:::rbd_id.image_client.0-clone3:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:52.348992+0000", 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.13535979400000001, 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003286775, 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1934 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.348992+0000", 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.348992+0000", 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.348993+0000", 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2610000000000001e-06 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.348998+0000", 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6240000000000001e-06 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.349007+0000", 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2609999999999999e-06 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.349015+0000", 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2400000000000007e-06 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.349077+0000", 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1427000000000007e-05 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.349081+0000", 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9890000000000003e-06 2024-09-18T01:02:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.350959+0000", 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018780579999999999 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.351029+0000", 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0038999999999998e-05 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.351129+0000", 2024-09-18T01:02:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9964999999999996e-05 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.351681+0000", 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00055164199999999997 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.352209+0000", 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052822199999999996 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.352240+0000", 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1163e-05 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.352279+0000", 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8884000000000002e-05 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4566.0:3551 3.17 e31/28 3:ed2b1a65:::rbd_data.1103b20ac255.0000000000000009:head v 31'39, mlcod=30'35)", 2024-09-18T01:02:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:52.406011+0000", 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.078341071999999998, 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003494951, 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406011+0000", 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406011+0000", 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406012+0000", 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8299999999999996e-07 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406018+0000", 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9149999999999999e-06 2024-09-18T01:02:52.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406032+0000", 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3969e-05 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406040+0000", 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9189999999999999e-06 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406093+0000", 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3162999999999999e-05 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.406115+0000", 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1872e-05 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.409473+0000", 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033573449999999999 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:52.409506+0000", 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2984999999999997e-05 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:52.614 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:52.615 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:52.613+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:52.616 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:02:52.692 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-18T01:02:52.717 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:52.715+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:52.723 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:52.723 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:52.723 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:02:52.723 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:52.732 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:02:52.818 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:52.817+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:52.845 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:52.845 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:52.845 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:52.845 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:52.845 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:52.855 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:02:52.919 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:52.916+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:52.965 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:52.965 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:52.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:52.965 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:52.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.624976556, 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0076954079999999999, 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 37 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.966 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337899+0000", 2024-09-18T01:02:52.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337902+0000", 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6599999999999999e-06 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337905+0000", 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.371e-06 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.337913+0000", 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1580000000000005e-06 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341291+0000", 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033772979999999999 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341344+0000", 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3483999999999998e-05 2024-09-18T01:02:52.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.341426+0000", 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1462e-05 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.343020+0000", 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015945740000000001 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345556+0000", 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025358749999999999 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345578+0000", 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1865000000000002e-05 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.345594+0000", 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6430999999999999e-05 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.971 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.430968974999999, 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077811820000000002, 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:52.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 111 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:52.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531907+0000", 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6600000000000002e-07 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531911+0000", 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4929999999999998e-06 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531917+0000", 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5509999999999996e-06 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531934+0000", 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6356000000000001e-05 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535192+0000", 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032581369999999999 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535241+0000", 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9363999999999997e-05 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535318+0000", 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7078999999999998e-05 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538191+0000", 2024-09-18T01:02:52.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0028732240000000002 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539619+0000", 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014276619999999999 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539661+0000", 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1819000000000001e-05 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:52.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539688+0000", 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7030999999999999e-05 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.430926809999999, 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077932879999999998, 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.978 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 112 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:52.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.531949+0000", 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6100000000000001e-07 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532030+0000", 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1082999999999996e-05 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532040+0000", 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9029999999999999e-06 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532047+0000", 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535639+0000", 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0035927009999999998 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535685+0000", 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5661000000000002e-05 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.535757+0000", 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2113999999999994e-05 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538221+0000", 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024644440000000001 2024-09-18T01:02:52.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539694+0000", 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001472283 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539719+0000", 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4777999999999999e-05 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539742+0000", 2024-09-18T01:02:52.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3547e-05 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.430815776999999, 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0077280339999999999, 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:52.984 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 113 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532060+0000", 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532061+0000", 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1900000000000002e-07 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532064+0000", 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8720000000000004e-06 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532078+0000", 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3766e-05 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532085+0000", 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.939e-06 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536066+0000", 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039811229999999996 2024-09-18T01:02:52.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536113+0000", 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6292000000000003e-05 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536186+0000", 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3632000000000002e-05 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538252+0000", 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020659419999999999 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539746+0000", 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014934670000000001 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539770+0000", 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4311000000000002e-05 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539788+0000", 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7971000000000001e-05 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.430779381000001, 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.007737715, 2024-09-18T01:02:52.990 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 114 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532096+0000", 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532097+0000", 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3499999999999996e-07 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532102+0000", 2024-09-18T01:02:52.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8770000000000001e-06 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532108+0000", 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.117e-06 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532118+0000", 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0101999999999999e-05 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:52.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536401+0000", 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042836200000000001 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536448+0000", 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.638e-05 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536522+0000", 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3769000000000005e-05 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538275+0000", 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017533819999999999 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539792+0000", 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001516861 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539814+0000", 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2305000000000001e-05 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.539834+0000", 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9766999999999999e-05 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.430737542999999, 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010209316, 2024-09-18T01:02:52.996 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 115 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532138+0000", 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0900000000000002e-07 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532186+0000", 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7521999999999997e-05 2024-09-18T01:02:52.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532192+0000", 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5049999999999998e-06 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532195+0000", 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5159999999999999e-06 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536736+0000", 2024-09-18T01:02:52.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0045408999999999996 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536783+0000", 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6956999999999998e-05 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.536851+0000", 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8118000000000001e-05 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538944+0000", 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020932110000000002 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542279+0000", 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033348679999999999 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542321+0000", 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1520999999999998e-05 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542347+0000", 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6789000000000001e-05 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.430673550000002, 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010197308, 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.002 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 116 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532202+0000", 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532203+0000", 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3599999999999999e-07 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532205+0000", 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5610000000000001e-06 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532209+0000", 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9639999999999997e-06 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532213+0000", 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5939999999999998e-06 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537234+0000", 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050209440000000003 2024-09-18T01:02:53.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537281+0000", 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6955e-05 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537352+0000", 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1870000000000001e-05 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538964+0000", 2024-09-18T01:02:53.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001611517 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542353+0000", 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003389076 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542376+0000", 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3456e-05 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542399+0000", 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2935e-05 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.430658447999999, 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.010229594999999999, 2024-09-18T01:02:53.008 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 117 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532217+0000", 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.03e-07 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532220+0000", 2024-09-18T01:02:53.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1710000000000001e-06 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532223+0000", 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4740000000000001e-06 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.532227+0000", 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5920000000000001e-06 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537567+0000", 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053407589999999996 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537615+0000", 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7590999999999999e-05 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.537694+0000", 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9015000000000002e-05 2024-09-18T01:02:53.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.538995+0000", 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013009429999999999 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542403+0000", 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034077180000000001 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542428+0000", 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5607000000000001e-05 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:26.542447+0000", 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8422000000000001e-05 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:130 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:53.014 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958927972000001, 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0095908409999999993, 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 130 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003948+0000", 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0500000000000002e-07 2024-09-18T01:02:53.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003953+0000", 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2860000000000001e-06 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003964+0000", 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1259e-05 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003970+0000", 2024-09-18T01:02:53.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1349999999999997e-06 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004461+0000", 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00049077799999999998 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004507+0000", 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5677000000000003e-05 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004590+0000", 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3610999999999996e-05 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006628+0000", 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020381240000000001 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013439+0000", 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.006810633 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013498+0000", 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9101999999999998e-05 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013538+0000", 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0431e-05 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.020 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958889724999999, 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096617260000000007, 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 131 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003986+0000", 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.003987+0000", 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004027+0000", 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0516999999999998e-05 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004034+0000", 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5220000000000004e-06 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004038+0000", 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0960000000000003e-06 2024-09-18T01:02:53.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004894+0000", 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00085607399999999996 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004982+0000", 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7811000000000003e-05 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005115+0000", 2024-09-18T01:02:53.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013318600000000001 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006634+0000", 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001519643 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013579+0000", 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0069450140000000002 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013610+0000", 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0849999999999998e-05 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013648+0000", 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7372000000000001e-05 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.026 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958830969000001, 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096891579999999998, 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 132 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.027 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004045+0000", 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.15e-07 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004047+0000", 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.232e-06 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004051+0000", 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.58e-06 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004056+0000", 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0839999999999999e-06 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005528+0000", 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0014725350000000001 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005573+0000", 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4518999999999998e-05 2024-09-18T01:02:53.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.005648+0000", 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4757000000000003e-05 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007604+0000", 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019566739999999998 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013684+0000", 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0060797519999999999 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013712+0000", 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8214000000000001e-05 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013734+0000", 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1495999999999999e-05 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958708016999999, 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0096596410000000001, 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.033 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 133 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004168+0000", 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4099999999999998e-07 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004173+0000", 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4270000000000001e-06 2024-09-18T01:02:53.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004188+0000", 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5435999999999999e-05 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004198+0000", 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0034000000000001e-05 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006144+0000", 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019458489999999999 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006225+0000", 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1207999999999999e-05 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006346+0000", 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012035400000000001 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008493+0000", 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021473849999999999 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013769+0000", 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052760510000000004 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013795+0000", 2024-09-18T01:02:53.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6205000000000001e-05 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013827+0000", 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1951e-05 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958662397000001, 2024-09-18T01:02:53.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097221809999999999, 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 134 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:53.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004213+0000", 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004214+0000", 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2200000000000004e-07 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004280+0000", 2024-09-18T01:02:53.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5707e-05 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004291+0000", 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.154e-05 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004306+0000", 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5088e-05 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006681+0000", 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002374953 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006754+0000", 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2490000000000006e-05 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.006825+0000", 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1667000000000003e-05 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009234+0000", 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002408477 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013872+0000", 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0046381599999999997 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013906+0000", 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3878999999999999e-05 2024-09-18T01:02:53.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013935+0000", 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9598e-05 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958553654999999, 2024-09-18T01:02:53.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097082049999999993, 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 135 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:53.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004322+0000", 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004323+0000", 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004328+0000", 2024-09-18T01:02:53.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5559999999999998e-06 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004338+0000", 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9699999999999994e-06 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004344+0000", 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4009999999999996e-06 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007275+0000", 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002931309 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007321+0000", 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6292000000000003e-05 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007394+0000", 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2354000000000006e-05 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010084+0000", 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0026903740000000002 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013970+0000", 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0038859210000000001 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.013997+0000", 2024-09-18T01:02:53.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.669e-05 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014030+0000", 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.345e-05 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958522326000001, 2024-09-18T01:02:53.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0097521970000000006, 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 136 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004353+0000", 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004354+0000", 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7800000000000001e-07 2024-09-18T01:02:53.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004360+0000", 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9660000000000001e-06 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004370+0000", 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0550999999999999e-05 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004374+0000", 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0500000000000002e-06 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007634+0000", 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003259498 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007684+0000", 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9768000000000002e-05 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.007758+0000", 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4017999999999995e-05 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010088+0000", 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0023301569999999998 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014061+0000", 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039732439999999999 2024-09-18T01:02:53.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014087+0000", 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.618e-05 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014105+0000", 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8187000000000001e-05 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958491142, 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098164900000000006, 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 137 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004384+0000", 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004385+0000", 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:02:53.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004432+0000", 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7184000000000001e-05 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004437+0000", 2024-09-18T01:02:53.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1730000000000003e-06 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004444+0000", 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9310000000000002e-06 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008052+0000", 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036079279999999998 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008097+0000", 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5011e-05 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008170+0000", 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2904999999999994e-05 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010090+0000", 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001920274 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014138+0000", 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040482019999999999 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014166+0000", 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7657999999999998e-05 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014201+0000", 2024-09-18T01:02:53.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4888000000000002e-05 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958419022000001, 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098169759999999998, 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.064 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 138 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004457+0000", 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1600000000000002e-07 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004460+0000", 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2499999999999998e-06 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004464+0000", 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3239999999999996e-06 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004467+0000", 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8289999999999999e-06 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008528+0000", 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040605679999999996 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008574+0000", 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6492999999999997e-05 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008645+0000", 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0403999999999999e-05 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010093+0000", 2024-09-18T01:02:53.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001447802 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014230+0000", 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0041371280000000003 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014254+0000", 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4757e-05 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014274+0000", 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9105000000000001e-05 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958404032000001, 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0098886330000000008, 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.070 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 139 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004472+0000", 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004475+0000", 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8720000000000002e-06 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004479+0000", 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1060000000000002e-06 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004482+0000", 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7369999999999998e-06 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008917+0000", 2024-09-18T01:02:53.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.004435184 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.008966+0000", 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9592000000000002e-05 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009037+0000", 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0797999999999995e-05 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010095+0000", 2024-09-18T01:02:53.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0010581399999999999 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014302+0000", 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042062189999999998 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014328+0000", 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6432e-05 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.014360+0000", 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2190000000000002e-05 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958390438999999, 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.012201405, 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.076 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 140 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004485+0000", 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-07 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004529+0000", 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3432e-05 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004533+0000", 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5249999999999999e-06 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004537+0000", 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8160000000000004e-06 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009263+0000", 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0047261509999999996 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009311+0000", 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7568000000000003e-05 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009381+0000", 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0419000000000005e-05 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010098+0000", 2024-09-18T01:02:53.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00071670299999999996 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016625+0000", 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0065267939999999998 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016651+0000", 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5959999999999999e-05 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016687+0000", 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5772000000000002e-05 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4177.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 25.958333047, 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.012284938, 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.082 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4177", 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.98:0/16829169", 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 141 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004543+0000", 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2300000000000002e-07 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004546+0000", 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.864e-06 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004552+0000", 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1999999999999999e-06 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.004555+0000", 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.706e-06 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009757+0000", 2024-09-18T01:02:53.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052024180000000003 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009806+0000", 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8653999999999999e-05 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.009879+0000", 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3551000000000001e-05 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.010335+0000", 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00045522899999999997 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016767+0000", 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0064324079999999997 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016790+0000", 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3468999999999998e-05 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:27.016828+0000", 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7116000000000002e-05 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.088 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:53.090 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:02:53.091 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:53.087+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:669 read 0x3237000 thru 0x3240867 (0x9868 bytes) 2024-09-18T01:02:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:672 punch from 0xa758600 to 0xa75b000, (0x2a00 bytes) 2024-09-18T01:02:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:677 trunc from 0xb13da00 to 0xabd3800 2024-09-18T01:02:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:678 write 0x9588e00 thru 0x958b39a (0x259b bytes) 2024-09-18T01:02:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:679 punch from 0x7e73c00 to 0x7e7ca00, (0x8e00 bytes) 2024-09-18T01:02:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:680 write 0xa6f4400 thru 0xa6fc450 (0x8051 bytes) 2024-09-18T01:02:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:681 trunc from 0xabd3800 to 0x8fdaa00 2024-09-18T01:02:53.108 INFO:teuthology.orchestra.run.smithi164.stdout:684 read 0x594ec00 thru 0x595a50f (0xb910 bytes) 2024-09-18T01:02:53.108 INFO:teuthology.orchestra.run.smithi164.stdout:686 read 0x83a0000 thru 0x83a4d4a (0x4d4b bytes) 2024-09-18T01:02:53.109 INFO:teuthology.orchestra.run.smithi164.stdout:687 write 0x8fb2600 thru 0x8fbb299 (0x8c9a bytes) 2024-09-18T01:02:53.116 INFO:teuthology.orchestra.run.smithi164.stdout:689 trunc from 0x8fdaa00 to 0x3375800 2024-09-18T01:02:53.141 INFO:teuthology.orchestra.run.smithi164.stdout:690 punch from 0x1de2800 to 0x1de4400, (0x1c00 bytes) 2024-09-18T01:02:53.147 INFO:teuthology.orchestra.run.smithi164.stdout:694 trunc from 0x3375800 to 0x6fa1400 2024-09-18T01:02:53.156 INFO:teuthology.orchestra.run.smithi164.stdout:696 write 0x5781800 thru 0x5782b19 (0x131a bytes) 2024-09-18T01:02:53.176 INFO:teuthology.orchestra.run.smithi164.stdout:698 write 0xcfaa200 thru 0xcfb87ab (0xe5ac bytes) 2024-09-18T01:02:53.177 INFO:teuthology.orchestra.run.smithi164.stdout:700 write 0x22d9e00 thru 0x22e2a3c (0x8c3d bytes) 2024-09-18T01:02:53.178 INFO:teuthology.orchestra.run.smithi164.stdout:702 read 0x645f400 thru 0x646e236 (0xee37 bytes) 2024-09-18T01:02:53.178 INFO:teuthology.orchestra.run.smithi164.stdout:703 punch from 0x363a400 to 0x363d800, (0x3400 bytes) 2024-09-18T01:02:53.178 INFO:teuthology.orchestra.run.smithi164.stdout:704 read 0x310dc00 thru 0x310e122 (0x523 bytes) 2024-09-18T01:02:53.179 INFO:teuthology.orchestra.run.smithi164.stdout:709 write 0x26ffa00 thru 0x27092bc (0x98bd bytes) 2024-09-18T01:02:53.183 INFO:teuthology.orchestra.run.smithi164.stdout:710 write 0xa801a00 thru 0xa8086de (0x6cdf bytes) 2024-09-18T01:02:53.183 INFO:teuthology.orchestra.run.smithi164.stdout:714 write 0x7eb7200 thru 0x7ebfbde (0x89df bytes) 2024-09-18T01:02:53.184 INFO:teuthology.orchestra.run.smithi164.stdout:716 punch from 0x4ef8c00 to 0x4efd600, (0x4a00 bytes) 2024-09-18T01:02:53.184 INFO:teuthology.orchestra.run.smithi164.stdout:717 trunc from 0xcfb8800 to 0xc833400 2024-09-18T01:02:53.190 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:53.188+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:53.211 INFO:teuthology.orchestra.run.smithi164.stdout:719 read 0x2220000 thru 0x22242de (0x42df bytes) 2024-09-18T01:02:53.212 INFO:teuthology.orchestra.run.smithi164.stdout:720 read 0x7f0d400 thru 0x7f0ea88 (0x1689 bytes) 2024-09-18T01:02:53.212 INFO:teuthology.orchestra.run.smithi164.stdout:723 read 0x1d61400 thru 0x1d6a38f (0x8f90 bytes) 2024-09-18T01:02:53.213 INFO:teuthology.orchestra.run.smithi164.stdout:724 read 0x8961a00 thru 0x8970bfd (0xf1fe bytes) 2024-09-18T01:02:53.213 INFO:teuthology.orchestra.run.smithi164.stdout:725 read 0x552c000 thru 0x553ab3a (0xeb3b bytes) 2024-09-18T01:02:53.213 INFO:teuthology.orchestra.run.smithi164.stdout:727 read 0x1229400 thru 0x1236ea8 (0xdaa9 bytes) 2024-09-18T01:02:53.213 INFO:teuthology.orchestra.run.smithi164.stdout:728 punch from 0xb809200 to 0xb810c00, (0x7a00 bytes) 2024-09-18T01:02:53.213 INFO:teuthology.orchestra.run.smithi164.stdout:732 write 0x759ba00 thru 0x75aab87 (0xf188 bytes) 2024-09-18T01:02:53.214 INFO:teuthology.orchestra.run.smithi164.stdout:733 write 0x6a2d600 thru 0x6a313aa (0x3dab bytes) 2024-09-18T01:02:53.221 INFO:teuthology.orchestra.run.smithi164.stdout:740 trunc from 0xc833400 to 0x61ab800 2024-09-18T01:02:53.252 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:53.252 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:53.252 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:02:53.252 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:53.261 INFO:teuthology.orchestra.run.smithi164.stdout:741 write 0xccc2800 thru 0xcccb915 (0x9116 bytes) 2024-09-18T01:02:53.262 INFO:teuthology.orchestra.run.smithi164.stdout:747 write 0x8d59600 thru 0x8d62ce3 (0x96e4 bytes) 2024-09-18T01:02:53.262 INFO:teuthology.orchestra.run.smithi164.stdout:749 read 0xadf3c00 thru 0xae0031d (0xc71e bytes) 2024-09-18T01:02:53.262 INFO:teuthology.orchestra.run.smithi164.stdout:751 write 0x237f600 thru 0x2381a68 (0x2469 bytes) 2024-09-18T01:02:53.263 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:02:53.270 INFO:teuthology.orchestra.run.smithi164.stdout:753 trunc from 0xcccba00 to 0x5e7f000 2024-09-18T01:02:53.286 INFO:teuthology.orchestra.run.smithi164.stdout:754 punch from 0x2220a00 to 0x2228e00, (0x8400 bytes) 2024-09-18T01:02:53.290 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:53.289+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:53.296 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:02:53.296 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:02:53.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:02:53.300 INFO:teuthology.orchestra.run.smithi164.stdout:757 write 0x9303800 thru 0x93067d3 (0x2fd4 bytes) 2024-09-18T01:02:53.310 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:02:53.310 INFO:teuthology.orchestra.run.smithi164.stdout:758 read 0x2975c00 thru 0x297b586 (0x5987 bytes) 2024-09-18T01:02:53.326 INFO:teuthology.orchestra.run.smithi164.stdout:759 write 0xb3b0800 thru 0xb3bda50 (0xd251 bytes) 2024-09-18T01:02:53.333 INFO:teuthology.orchestra.run.smithi164.stdout:760 punch from 0x57e4800 to 0x57eba00, (0x7200 bytes) 2024-09-18T01:02:53.335 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xedb3600 2024-09-18T01:02:53.339 INFO:teuthology.orchestra.run.smithi164.stdout:762 trunc from 0xb3bdc00 to 0xedb3600 2024-09-18T01:02:53.349 INFO:teuthology.orchestra.run.smithi164.stdout:763 punch from 0xcfa6a00 to 0xcfac600, (0x5c00 bytes) 2024-09-18T01:02:53.357 INFO:teuthology.orchestra.run.smithi164.stdout:764 write 0x5e12400 thru 0x5e1f6d1 (0xd2d2 bytes) 2024-09-18T01:02:53.364 INFO:teuthology.orchestra.run.smithi164.stdout:766 trunc from 0xedb3600 to 0x8c94000 2024-09-18T01:02:53.382 INFO:teuthology.orchestra.run.smithi164.stdout:767 read 0x2f08400 thru 0x2f123b1 (0x9fb2 bytes) 2024-09-18T01:02:53.383 INFO:teuthology.orchestra.run.smithi164.stdout:768 read 0x5fc0a00 thru 0x5fc21d1 (0x17d2 bytes) 2024-09-18T01:02:53.383 INFO:teuthology.orchestra.run.smithi164.stdout:769 write 0x67e7600 thru 0x67ed174 (0x5b75 bytes) 2024-09-18T01:02:53.385 INFO:teuthology.orchestra.run.smithi164.stdout:772 trunc from 0x8c94000 to 0xa8cfc00 2024-09-18T01:02:53.391 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:53.390+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:53.396 INFO:teuthology.orchestra.run.smithi164.stdout:774 read 0x33b3a00 thru 0x33bc874 (0x8e75 bytes) 2024-09-18T01:02:53.397 INFO:teuthology.orchestra.run.smithi164.stdout:775 read 0x9a60000 thru 0x9a687ca (0x87cb bytes) 2024-09-18T01:02:53.397 INFO:teuthology.orchestra.run.smithi164.stdout:778 punch from 0x616dc00 to 0x6176000, (0x8400 bytes) 2024-09-18T01:02:53.399 INFO:teuthology.orchestra.run.smithi164.stdout:779 write 0x146f400 thru 0x1478277 (0x8e78 bytes) 2024-09-18T01:02:53.399 INFO:teuthology.orchestra.run.smithi164.stdout:781 punch from 0x1717800 to 0x171f800, (0x8000 bytes) 2024-09-18T01:02:53.400 INFO:teuthology.orchestra.run.smithi164.stdout:785 punch from 0x7784e00 to 0x778cc00, (0x7e00 bytes) 2024-09-18T01:02:53.400 INFO:teuthology.orchestra.run.smithi164.stdout:786 punch from 0x3829000 to 0x382e600, (0x5600 bytes) 2024-09-18T01:02:53.400 INFO:teuthology.orchestra.run.smithi164.stdout:787 punch from 0x175b200 to 0x1762600, (0x7400 bytes) 2024-09-18T01:02:53.400 INFO:teuthology.orchestra.run.smithi164.stdout:788 trunc from 0xa8cfc00 to 0xa375a00 2024-09-18T01:02:53.422 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:53.423 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:53.423 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:53.423 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:53.423 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:53.424 INFO:teuthology.orchestra.run.smithi164.stdout:789 write 0x694ea00 thru 0x695aad2 (0xc0d3 bytes) 2024-09-18T01:02:53.432 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:02:53.449 INFO:teuthology.orchestra.run.smithi164.stdout:791 write 0xe2cb800 thru 0xe2d0958 (0x5159 bytes) 2024-09-18T01:02:53.450 INFO:teuthology.orchestra.run.smithi164.stdout:792 read 0x37cc600 thru 0x37cd62c (0x102d bytes) 2024-09-18T01:02:53.450 INFO:teuthology.orchestra.run.smithi164.stdout:793 write 0x83ff800 thru 0x8403242 (0x3a43 bytes) 2024-09-18T01:02:53.459 INFO:teuthology.orchestra.run.smithi164.stdout:795 trunc from 0xe2d0a00 to 0x5c3be00 2024-09-18T01:02:53.492 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:53.491+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:53.497 INFO:teuthology.orchestra.run.smithi164.stdout:797 trunc from 0x5c3be00 to 0x1fc7000 2024-09-18T01:02:53.533 INFO:teuthology.orchestra.run.smithi164.stdout:798 write 0x67d3200 thru 0x67e1330 (0xe131 bytes) 2024-09-18T01:02:53.533 INFO:teuthology.orchestra.run.smithi164.stdout:799 read 0x4443400 thru 0x4445907 (0x2508 bytes) 2024-09-18T01:02:53.533 INFO:teuthology.orchestra.run.smithi164.stdout:801 write 0xafdc00 thru 0xb0b86c (0xdc6d bytes) 2024-09-18T01:02:53.538 INFO:teuthology.orchestra.run.smithi164.stdout:802 read 0xe8dc00 thru 0xe95eb6 (0x82b7 bytes) 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.470223+0000", 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 10.077142421, 2024-09-18T01:02:53.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0061150029999999999, 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 9 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470223+0000", 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470223+0000", 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470224+0000", 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3349999999999999e-06 2024-09-18T01:02:53.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470237+0000", 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3030999999999999e-05 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470248+0000", 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1022999999999999e-05 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470255+0000", 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2710000000000003e-06 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470300+0000", 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4777999999999997e-05 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:53.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.470307+0000", 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4160000000000001e-06 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.473645+0000", 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0033382479999999998 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.473713+0000", 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8299000000000005e-05 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.473798+0000", 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5017000000000003e-05 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.474959+0000", 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0011607900000000001 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.476275+0000", 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013163700000000001 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.476311+0000", 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6127999999999997e-05 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.476338+0000", 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6296999999999999e-05 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.557 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:720 3.3 3:c542c01e:::rbd_data.1103b05c2cf1.0000000000000024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.428965+0000", 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.1183996729999999, 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0048215059999999997, 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.558 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 720 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428965+0000", 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428965+0000", 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428966+0000", 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428971+0000", 2024-09-18T01:02:53.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2220000000000004e-06 2024-09-18T01:02:53.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428983+0000", 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2877999999999999e-05 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.428990+0000", 2024-09-18T01:02:53.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9650000000000002e-06 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429044+0000", 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3310999999999998e-05 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429124+0000", 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0663999999999999e-05 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429262+0000", 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000138138 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432562+0000", 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032997880000000001 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.433718+0000", 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0011555090000000001 2024-09-18T01:02:53.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.433750+0000", 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1831000000000001e-05 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.433787+0000", 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7385999999999997e-05 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:710 3.1b e26/20 3:d93a887d:::rbd_data.1103b05c2cf1.000000000000001d:head v 26'87, mlcod=26'85)", 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429361+0000", 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.1180039730000004, 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032690079999999999, 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429361+0000", 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429361+0000", 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429362+0000", 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429367+0000", 2024-09-18T01:02:53.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1869999999999996e-06 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429377+0000", 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9410000000000002e-06 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429385+0000", 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7030000000000006e-06 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429424+0000", 2024-09-18T01:02:53.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9208000000000001e-05 2024-09-18T01:02:53.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429446+0000", 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1878e-05 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432590+0000", 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031443370000000001 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432630+0000", 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0147e-05 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.570 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:719 3.1 e26/20 3:855394e7:::rbd_data.1103b05c2cf1.0000000000000023:head v 26'6, mlcod=26'4)", 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.1179661940000001, 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003258295, 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429399+0000", 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0100000000000001e-07 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429403+0000", 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3110000000000001e-06 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429410+0000", 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4200000000000001e-06 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429414+0000", 2024-09-18T01:02:53.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1500000000000001e-06 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429471+0000", 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6928000000000001e-05 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429490+0000", 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8627e-05 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432606+0000", 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031164560000000001 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432657+0000", 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1102e-05 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.575 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:714 3.19 e26/20 3:9c24ad54:::rbd_data.1103b05c2cf1.0000000000000020:head v 26'101, mlcod=26'99)", 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429597+0000", 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.1177679170000001, 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0030934769999999999, 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429597+0000", 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429597+0000", 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429598+0000", 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6700000000000003e-07 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429604+0000", 2024-09-18T01:02:53.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1360000000000002e-06 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429614+0000", 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5529999999999998e-06 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429620+0000", 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9859999999999999e-06 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429665+0000", 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5389999999999997e-05 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429683+0000", 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8056000000000001e-05 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432659+0000", 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002976228 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432691+0000", 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1461000000000003e-05 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:760 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'98, mlcod=26'88)", 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.445182+0000", 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.1021831520000003, 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050740170000000001, 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.580 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445182+0000", 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445182+0000", 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445183+0000", 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2799999999999996e-07 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445186+0000", 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4130000000000002e-06 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445193+0000", 2024-09-18T01:02:53.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9469999999999997e-06 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445197+0000", 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7220000000000001e-06 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445268+0000", 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1180000000000006e-05 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.445286+0000", 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8346e-05 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450215+0000", 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049289410000000001 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450256+0000", 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0939999999999998e-05 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:761 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'99, mlcod=26'88)", 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.1013624990000004, 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.004282358, 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446003+0000", 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2500000000000001e-07 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446006+0000", 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446012+0000", 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1020000000000002e-06 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446016+0000", 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2200000000000001e-06 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446049+0000", 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3358000000000001e-05 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446063+0000", 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3813e-05 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450258+0000", 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0041955600000000001 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450285+0000", 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6588999999999999e-05 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.589 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:762 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'100, mlcod=26'88)", 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.1006893760000001, 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036332729999999998, 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446676+0000", 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8000000000000002e-07 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446679+0000", 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446684+0000", 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2650000000000004e-06 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446687+0000", 2024-09-18T01:02:53.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.88e-06 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446732+0000", 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5349000000000002e-05 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.446751+0000", 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8719000000000002e-05 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450286+0000", 2024-09-18T01:02:53.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0035353849999999998 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450309+0000", 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2874000000000002e-05 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:765 3.1d e26/20 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head v 26'3, mlcod=26'1)", 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447708+0000", 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.0996571470000003, 2024-09-18T01:02:53.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052494550000000001, 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447708+0000", 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447708+0000", 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447709+0000", 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1399999999999996e-07 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447715+0000", 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2360000000000001e-06 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447727+0000", 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1894e-05 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447734+0000", 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.002e-06 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447786+0000", 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2216999999999998e-05 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447808+0000", 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1682e-05 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452916+0000", 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051077450000000003 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452957+0000", 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1865e-05 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.598 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:763 3.1b e26/20 3:daf932aa:::rbd_object_map.1103b05c2cf1:head v 26'101, mlcod=26'95)", 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.09942853, 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.005068571, 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447937+0000", 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1500000000000004e-07 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447946+0000", 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2819999999999996e-06 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447956+0000", 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9590000000000007e-06 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447962+0000", 2024-09-18T01:02:53.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6420000000000001e-06 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448069+0000", 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010698 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448087+0000", 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8104000000000001e-05 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452970+0000", 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0048830820000000004 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453005+0000", 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4807000000000001e-05 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:770 3.1 e26/20 3:85d4f36e:::rbd_data.1103b05c2cf1.0000000000000014:head v 26'8, mlcod=26'6)", 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447979+0000", 2024-09-18T01:02:53.603 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.0993859710000002, 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050070510000000002, 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447979+0000", 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447979+0000", 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447980+0000", 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447985+0000", 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1139999999999997e-06 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447992+0000", 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0960000000000003e-06 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447997+0000", 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9849999999999997e-06 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448045+0000", 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8541000000000002e-05 2024-09-18T01:02:53.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448063+0000", 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8084000000000002e-05 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452948+0000", 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0048849430000000001 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.452986+0000", 2024-09-18T01:02:53.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7858000000000002e-05 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:773 3.3 3:c65d1f6d:::rbd_data.1103b05c2cf1.0000000000000017:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.450741+0000", 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.0966240960000002, 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0044119839999999999, 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.608 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 773 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450741+0000", 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450741+0000", 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450742+0000", 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0279999999999999e-06 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450747+0000", 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8369999999999996e-06 2024-09-18T01:02:53.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450757+0000", 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0043e-05 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450762+0000", 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450807+0000", 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4376999999999999e-05 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450884+0000", 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7440000000000004e-05 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450996+0000", 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111579 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453291+0000", 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0022950750000000002 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.455096+0000", 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0018052980000000001 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.455127+0000", 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0642999999999998e-05 2024-09-18T01:02:53.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.455153+0000", 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6429000000000001e-05 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:830 3.1a 3:5f37e58f:::rbd_data.1103b05c2cf1.0000000000000018:head [stat,write 515584~8704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.487225+0000", 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 6.0601403850000004, 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.007291563, 2024-09-18T01:02:53.614 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 830 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487225+0000", 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487225+0000", 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487226+0000", 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.54e-07 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487247+0000", 2024-09-18T01:02:53.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1080000000000001e-05 2024-09-18T01:02:53.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487259+0000", 2024-09-18T01:02:53.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.201e-05 2024-09-18T01:02:53.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487264+0000", 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2889999999999997e-06 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487317+0000", 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3007999999999998e-05 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.487323+0000", 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0209999999999996e-06 2024-09-18T01:02:53.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.489522+0000", 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002198565 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.489574+0000", 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1956000000000002e-05 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:53.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.489715+0000", 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014140999999999999 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.490363+0000", 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064768300000000003 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.494457+0000", 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0040941229999999999 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.494490+0000", 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2891000000000001e-05 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.494516+0000", 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6673e-05 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.621 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:961 3.3 3:c33c4291:::rbd_data.1103b05c2cf1.00000000000000fa:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.620802+0000", 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 5.9265627729999997, 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0045840669999999998, 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 961 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620802+0000", 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620802+0000", 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620804+0000", 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2810000000000001e-06 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620808+0000", 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4299999999999999e-06 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620822+0000", 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4287000000000001e-05 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620834+0000", 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1747e-05 2024-09-18T01:02:53.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620889+0000", 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4425000000000002e-05 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.620977+0000", 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.8243000000000002e-05 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:53.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.621088+0000", 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111494 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.621657+0000", 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00056824299999999996 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.625295+0000", 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036382680000000001 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.625347+0000", 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1882999999999998e-05 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.625386+0000", 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9765999999999998e-05 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:1106 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T01:02:53.627 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:49.342546+0000", 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.204819305, 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0032088400000000001, 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1106 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342546+0000", 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342546+0000", 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342547+0000", 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2600000000000002e-07 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342550+0000", 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3519999999999998e-06 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342558+0000", 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9200000000000004e-06 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342562+0000", 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6399999999999996e-06 2024-09-18T01:02:53.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342614+0000", 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1248000000000001e-05 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342707+0000", 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.323e-05 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:53.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.342944+0000", 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000237049 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.343755+0000", 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00081066600000000003 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.345639+0000", 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001884344 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.345722+0000", 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3153999999999997e-05 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:49.345755+0000", 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2511000000000001e-05 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.633 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:587 3.3 3:c171b8a0:::rbd_data.11031049ea29.00000000000000cf:head [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.183393+0000", 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.36397174999999998, 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031891689999999999, 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 587 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183393+0000", 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183393+0000", 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183394+0000", 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8700000000000005e-07 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183398+0000", 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0559999999999998e-06 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183407+0000", 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9209999999999998e-06 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183420+0000", 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2544999999999999e-05 2024-09-18T01:02:53.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183460+0000", 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0389000000000002e-05 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183500+0000", 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0120999999999998e-05 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.186583+0000", 2024-09-18T01:02:53.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0030823500000000002 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4608.0:588 3.11 e31/20 3:8d6b5fa9:::rbd_data.11031049ea29.0000000000000022:head v 31'4, mlcod=26'2)", 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.183914+0000", 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.36345163800000002, 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0041238749999999999, 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183914+0000", 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183914+0000", 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183914+0000", 2024-09-18T01:02:53.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8299999999999996e-07 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183932+0000", 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799e-05 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183941+0000", 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.0380000000000005e-06 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.183947+0000", 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1840000000000004e-06 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.184000+0000", 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3185e-05 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.184021+0000", 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0574999999999999e-05 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.187997+0000", 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039760309999999997 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.188037+0000", 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0380000000000003e-05 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.642 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:734 3.9 3:948ae561:::rbd_data.11031049ea29.00000000000000b8:head [set-alloc-hint object_size 1048576 write_size 1048576,write 37376~28160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.215342+0000", 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.33202309800000002, 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0035861790000000001, 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:53.643 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 734 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215342+0000", 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215342+0000", 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215343+0000", 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5999999999999991e-07 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215382+0000", 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9183000000000001e-05 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215393+0000", 2024-09-18T01:02:53.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1102e-05 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215399+0000", 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4790000000000004e-06 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215454+0000", 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5216000000000001e-05 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215577+0000", 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012305000000000001 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.215704+0000", 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000126556 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.218370+0000", 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0026668939999999999 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.218865+0000", 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00049406899999999998 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.218894+0000", 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9660999999999999e-05 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.218928+0000", 2024-09-18T01:02:53.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4008999999999999e-05 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:954 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.305480+0000", 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.24188544300000001, 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0041200560000000004, 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.649 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 954 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305480+0000", 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305480+0000", 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305481+0000", 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5300000000000003e-07 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305485+0000", 2024-09-18T01:02:53.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305496+0000", 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0679999999999999e-05 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305501+0000", 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6590000000000004e-06 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305554+0000", 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2731e-05 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.305561+0000", 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2080000000000003e-06 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.309486+0000", 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0039250780000000002 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.309538+0000", 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1513e-05 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.309600+0000", 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1799999999999998e-05 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1346 3.1c 3:3aeec32e:::rbd_data.11031049ea29.0000000000000084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.472708+0000", 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.074656921000000001, 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0057541099999999998, 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1346 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:53.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.472708+0000", 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.472708+0000", 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.472709+0000", 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:02:53.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.472712+0000", 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7769999999999998e-06 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.472723+0000", 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1469e-05 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.472729+0000", 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.474005+0000", 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001276613 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.474058+0000", 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2089000000000002e-05 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.474142+0000", 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4281000000000002e-05 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.475182+0000", 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0010397480000000001 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.478376+0000", 2024-09-18T01:02:53.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031947350000000002 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.478423+0000", 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6714e-05 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.478462+0000", 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9271999999999999e-05 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:53.661 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:53.661 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:53.663 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:02:53.664 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:02:53.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:02:53.667 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-18T01:02:53.305882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16625072,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5738,"num_read_kb":5174,"num_write":854,"num_write_kb":5663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":81996,"num_keys_recovered":14,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":153848,"kb_used_data":2036,"kb_used_omap":7,"kb_used_meta":151672,"kb_avail":471705352,"statfs":{"total":483183820800,"available":483026280448,"internally_reserved":0,"allocated":2084864,"data_stored":2393388,"data_compressed":133178,"data_compressed_allocated":229376,"data_compressed_original":1581056,"omap_allocated":7973,"internal_metadata":155312347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16034620,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":5557,"num_read_kb":5022,"num_write":687,"num_write_kb":3239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":81996,"num_keys_recovered":14,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.958927"},"pg_stats":[{"pgid":"3.1f","version":"27'9","reported_seq":99,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.287051+0000","last_change":"2024-09-18T01:02:43.469611+0000","last_active":"2024-09-18T01:02:49.287051+0000","last_peered":"2024-09-18T01:02:49.287051+0000","last_clean":"2024-09-18T01:02:49.287051+0000","last_became_active":"2024-09-18T01:02:41.451184+0000","last_became_peered":"2024-09-18T01:02:41.451184+0000","last_unstale":"2024-09-18T01:02:49.287051+0000","last_undegraded":"2024-09-18T01:02:49.287051+0000","last_fullsized":"2024-09-18T01:02:49.287051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:40.900573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005167,"stat_sum":{"num_bytes":5327376,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":63,"num_read_kb":115,"num_write":9,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'12","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.287862+0000","last_change":"2024-09-18T01:02:43.469963+0000","last_active":"2024-09-18T01:02:49.287862+0000","last_peered":"2024-09-18T01:02:49.287862+0000","last_clean":"2024-09-18T01:02:49.287862+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:02:49.287862+0000","last_undegraded":"2024-09-18T01:02:49.287862+0000","last_fullsized":"2024-09-18T01:02:49.287862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:30:39.922945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013026730000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12,"num_write":12,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"26'3","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.322841+0000","last_change":"2024-09-18T01:02:49.322327+0000","last_active":"2024-09-18T01:02:49.322841+0000","last_peered":"2024-09-18T01:02:49.322841+0000","last_clean":"2024-09-18T01:02:49.322841+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:02:49.322841+0000","last_undegraded":"2024-09-18T01:02:49.322841+0000","last_fullsized":"2024-09-18T01:02:49.322841+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:06.006283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918460000000001,"stat_sum":{"num_bytes":393216,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"29'4","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.345723+0000","last_change":"2024-09-18T01:02:49.301865+0000","last_active":"2024-09-18T01:02:49.345723+0000","last_peered":"2024-09-18T01:02:49.345723+0000","last_clean":"2024-09-18T01:02:49.345723+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:02:49.345723+0000","last_undegraded":"2024-09-18T01:02:49.345723+0000","last_fullsized":"2024-09-18T01:02:49.345723+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:57:03.101259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242879999999999,"stat_sum":{"num_bytes":41,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"29'140","reported_seq":175,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.300835+0000","last_change":"2024-09-18T01:02:43.469715+0000","last_active":"2024-09-18T01:02:49.300835+0000","last_peered":"2024-09-18T01:02:49.300835+0000","last_clean":"2024-09-18T01:02:49.300835+0000","last_became_active":"2024-09-18T01:02:41.454655+0000","last_became_peered":"2024-09-18T01:02:41.454655+0000","last_unstale":"2024-09-18T01:02:49.300835+0000","last_undegraded":"2024-09-18T01:02:49.300835+0000","last_fullsized":"2024-09-18T01:02:49.300835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:44:53.616588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087094899999999999,"stat_sum":{"num_bytes":97,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":667,"num_read_kb":536,"num_write":233,"num_write_kb":350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"27'10","reported_seq":278,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.338180+0000","last_change":"2024-09-18T01:02:43.474216+0000","last_active":"2024-09-18T01:02:49.338180+0000","last_peered":"2024-09-18T01:02:49.338180+0000","last_clean":"2024-09-18T01:02:49.338180+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:02:49.338180+0000","last_undegraded":"2024-09-18T01:02:49.338180+0000","last_fullsized":"2024-09-18T01:02:49.338180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:40:35.122865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074574000000000005,"stat_sum":{"num_bytes":670227,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":237,"num_read_kb":232,"num_write":9,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"29'108","reported_seq":152,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.361732+0000","last_change":"2024-09-18T01:02:43.475997+0000","last_active":"2024-09-18T01:02:49.361732+0000","last_peered":"2024-09-18T01:02:49.361732+0000","last_clean":"2024-09-18T01:02:49.361732+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:02:49.361732+0000","last_undegraded":"2024-09-18T01:02:49.361732+0000","last_fullsized":"2024-09-18T01:02:49.361732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:59.655488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038270300000000003,"stat_sum":{"num_bytes":41,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":504,"num_read_kb":399,"num_write":168,"num_write_kb":234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"27'7","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.288566+0000","last_change":"2024-09-18T01:02:43.468721+0000","last_active":"2024-09-18T01:02:49.288566+0000","last_peered":"2024-09-18T01:02:49.288566+0000","last_clean":"2024-09-18T01:02:49.288566+0000","last_became_active":"2024-09-18T01:02:41.459321+0000","last_became_peered":"2024-09-18T01:02:41.459321+0000","last_unstale":"2024-09-18T01:02:49.288566+0000","last_undegraded":"2024-09-18T01:02:49.288566+0000","last_fullsized":"2024-09-18T01:02:49.288566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:28:49.053638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061623500000000002,"stat_sum":{"num_bytes":422912,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":8,"num_write":7,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"26'4","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.315232+0000","last_change":"2024-09-18T01:02:49.315232+0000","last_active":"2024-09-18T01:02:49.315232+0000","last_peered":"2024-09-18T01:02:49.315232+0000","last_clean":"2024-09-18T01:02:49.315232+0000","last_became_active":"2024-09-18T01:02:49.293095+0000","last_became_peered":"2024-09-18T01:02:49.293095+0000","last_unstale":"2024-09-18T01:02:49.315232+0000","last_undegraded":"2024-09-18T01:02:49.315232+0000","last_fullsized":"2024-09-18T01:02:49.315232+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:58.228891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097728599999999995,"stat_sum":{"num_bytes":655360,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":12,"num_write":4,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"29'5","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.351803+0000","last_change":"2024-09-18T01:02:49.351291+0000","last_active":"2024-09-18T01:02:49.351803+0000","last_peered":"2024-09-18T01:02:49.351803+0000","last_clean":"2024-09-18T01:02:49.351803+0000","last_became_active":"2024-09-18T01:02:49.291045+0000","last_became_peered":"2024-09-18T01:02:49.291045+0000","last_unstale":"2024-09-18T01:02:49.351803+0000","last_undegraded":"2024-09-18T01:02:49.351803+0000","last_fullsized":"2024-09-18T01:02:49.351803+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:49.614720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099035500000000005,"stat_sum":{"num_bytes":1880161,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"27'3","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.283907+0000","last_change":"2024-09-18T01:02:43.464371+0000","last_active":"2024-09-18T01:02:49.283907+0000","last_peered":"2024-09-18T01:02:49.283907+0000","last_clean":"2024-09-18T01:02:49.283907+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:02:49.283907+0000","last_undegraded":"2024-09-18T01:02:49.283907+0000","last_fullsized":"2024-09-18T01:02:49.283907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:47.421210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077757900000000003,"stat_sum":{"num_bytes":245760,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"27'12","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.283818+0000","last_change":"2024-09-18T01:02:43.464338+0000","last_active":"2024-09-18T01:02:49.283818+0000","last_peered":"2024-09-18T01:02:49.283818+0000","last_clean":"2024-09-18T01:02:49.283818+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:02:49.283818+0000","last_undegraded":"2024-09-18T01:02:49.283818+0000","last_fullsized":"2024-09-18T01:02:49.283818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:18:12.041100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080305299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":12,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.284207+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:02:49.284207+0000","last_peered":"2024-09-18T01:02:49.284207+0000","last_clean":"2024-09-18T01:02:49.284207+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:02:49.284207+0000","last_undegraded":"2024-09-18T01:02:49.284207+0000","last_fullsized":"2024-09-18T01:02:49.284207+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.286703+0000","last_change":"2024-09-18T01:02:43.469252+0000","last_active":"2024-09-18T01:02:49.286703+0000","last_peered":"2024-09-18T01:02:49.286703+0000","last_clean":"2024-09-18T01:02:49.286703+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:02:49.286703+0000","last_undegraded":"2024-09-18T01:02:49.286703+0000","last_fullsized":"2024-09-18T01:02:49.286703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:47.525091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066086499999999996,"stat_sum":{"num_bytes":262144,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.286992+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:02:49.286992+0000","last_peered":"2024-09-18T01:02:49.286992+0000","last_clean":"2024-09-18T01:02:49.286992+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:02:49.286992+0000","last_undegraded":"2024-09-18T01:02:49.286992+0000","last_fullsized":"2024-09-18T01:02:49.286992+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"27'5","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.288760+0000","last_change":"2024-09-18T01:02:43.469995+0000","last_active":"2024-09-18T01:02:49.288760+0000","last_peered":"2024-09-18T01:02:49.288760+0000","last_clean":"2024-09-18T01:02:49.288760+0000","last_became_active":"2024-09-18T01:02:41.456801+0000","last_became_peered":"2024-09-18T01:02:41.456801+0000","last_unstale":"2024-09-18T01:02:49.288760+0000","last_undegraded":"2024-09-18T01:02:49.288760+0000","last_fullsized":"2024-09-18T01:02:49.288760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:39:23.473487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159869,"stat_sum":{"num_bytes":437760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.284062+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:02:49.284062+0000","last_peered":"2024-09-18T01:02:49.284062+0000","last_clean":"2024-09-18T01:02:49.284062+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:02:49.284062+0000","last_undegraded":"2024-09-18T01:02:49.284062+0000","last_fullsized":"2024-09-18T01:02:49.284062+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"26'13","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.286028+0000","last_change":"2024-09-18T01:02:43.476334+0000","last_active":"2024-09-18T01:02:49.286028+0000","last_peered":"2024-09-18T01:02:49.286028+0000","last_clean":"2024-09-18T01:02:49.286028+0000","last_became_active":"2024-09-18T01:02:41.457022+0000","last_became_peered":"2024-09-18T01:02:41.457022+0000","last_unstale":"2024-09-18T01:02:49.286028+0000","last_undegraded":"2024-09-18T01:02:49.286028+0000","last_fullsized":"2024-09-18T01:02:49.286028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:47:27.236083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050705100000000001,"stat_sum":{"num_bytes":393216,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":13,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.286157+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:02:49.286157+0000","last_peered":"2024-09-18T01:02:49.286157+0000","last_clean":"2024-09-18T01:02:49.286157+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:02:49.286157+0000","last_undegraded":"2024-09-18T01:02:49.286157+0000","last_fullsized":"2024-09-18T01:02:49.286157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.295846+0000","last_change":"2024-09-18T01:02:49.294263+0000","last_active":"2024-09-18T01:02:49.295846+0000","last_peered":"2024-09-18T01:02:49.295846+0000","last_clean":"2024-09-18T01:02:49.295846+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:02:49.295846+0000","last_undegraded":"2024-09-18T01:02:49.295846+0000","last_fullsized":"2024-09-18T01:02:49.295846+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:23:18.872784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"27'102","reported_seq":1202,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.289084+0000","last_change":"2024-09-18T01:02:43.473795+0000","last_active":"2024-09-18T01:02:49.289084+0000","last_peered":"2024-09-18T01:02:49.289084+0000","last_clean":"2024-09-18T01:02:49.289084+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:02:49.289084+0000","last_undegraded":"2024-09-18T01:02:49.289084+0000","last_fullsized":"2024-09-18T01:02:49.289084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:45:07.252468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031347599999999998,"stat_sum":{"num_bytes":262144,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2220,"num_read_kb":1931,"num_write":56,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.331878+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:02:49.331878+0000","last_peered":"2024-09-18T01:02:49.331878+0000","last_clean":"2024-09-18T01:02:49.331878+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:02:49.331878+0000","last_undegraded":"2024-09-18T01:02:49.331878+0000","last_fullsized":"2024-09-18T01:02:49.331878+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"29'12","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.357108+0000","last_change":"2024-09-18T01:02:43.476102+0000","last_active":"2024-09-18T01:02:49.357108+0000","last_peered":"2024-09-18T01:02:49.357108+0000","last_clean":"2024-09-18T01:02:49.357108+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:02:49.357108+0000","last_undegraded":"2024-09-18T01:02:49.357108+0000","last_fullsized":"2024-09-18T01:02:49.357108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:47.165648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043811300000000001,"stat_sum":{"num_bytes":393232,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":125,"num_write":12,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.288034+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:02:49.288034+0000","last_peered":"2024-09-18T01:02:49.288034+0000","last_clean":"2024-09-18T01:02:49.288034+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:02:49.288034+0000","last_undegraded":"2024-09-18T01:02:49.288034+0000","last_fullsized":"2024-09-18T01:02:49.288034+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"26'8","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.285601+0000","last_change":"2024-09-18T01:02:43.468762+0000","last_active":"2024-09-18T01:02:49.285601+0000","last_peered":"2024-09-18T01:02:49.285601+0000","last_clean":"2024-09-18T01:02:49.285601+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:02:49.285601+0000","last_undegraded":"2024-09-18T01:02:49.285601+0000","last_fullsized":"2024-09-18T01:02:49.285601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:14:39.693782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296199999999997,"stat_sum":{"num_bytes":381952,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":42,"num_write":7,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.283802+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:02:49.283802+0000","last_peered":"2024-09-18T01:02:49.283802+0000","last_clean":"2024-09-18T01:02:49.283802+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:02:49.283802+0000","last_undegraded":"2024-09-18T01:02:49.283802+0000","last_fullsized":"2024-09-18T01:02:49.283802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"29'86","reported_seq":1194,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.339448+0000","last_change":"2024-09-18T01:02:49.337150+0000","last_active":"2024-09-18T01:02:49.339448+0000","last_peered":"2024-09-18T01:02:49.339448+0000","last_clean":"2024-09-18T01:02:49.339448+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:02:49.339448+0000","last_undegraded":"2024-09-18T01:02:49.339448+0000","last_fullsized":"2024-09-18T01:02:49.339448+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:21:22.827271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049608300000000003,"stat_sum":{"num_bytes":393232,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1826,"num_read_kb":1566,"num_write":49,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.309309+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:02:49.309309+0000","last_peered":"2024-09-18T01:02:49.309309+0000","last_clean":"2024-09-18T01:02:49.309309+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:02:49.309309+0000","last_undegraded":"2024-09-18T01:02:49.309309+0000","last_fullsized":"2024-09-18T01:02:49.309309+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":193,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.283802+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:02:49.283802+0000","last_peered":"2024-09-18T01:02:49.283802+0000","last_clean":"2024-09-18T01:02:49.283802+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:02:49.283802+0000","last_undegraded":"2024-09-18T01:02:49.283802+0000","last_fullsized":"2024-09-18T01:02:49.283802+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"25'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.288695+0000","last_change":"2024-09-18T01:02:43.468997+0000","last_active":"2024-09-18T01:02:49.288695+0000","last_peered":"2024-09-18T01:02:49.288695+0000","last_clean":"2024-09-18T01:02:49.288695+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:02:49.288695+0000","last_undegraded":"2024-09-18T01:02:49.288695+0000","last_fullsized":"2024-09-18T01:02:49.288695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063056400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.288255+0000","last_change":"2024-09-18T01:02:43.474261+0000","last_active":"2024-09-18T01:02:49.288255+0000","last_peered":"2024-09-18T01:02:49.288255+0000","last_clean":"2024-09-18T01:02:49.288255+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:02:49.288255+0000","last_undegraded":"2024-09-18T01:02:49.288255+0000","last_fullsized":"2024-09-18T01:02:49.288255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:05:07.519803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065863900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"29'12","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.283908+0000","last_change":"2024-09-18T01:02:43.464113+0000","last_active":"2024-09-18T01:02:49.283908+0000","last_peered":"2024-09-18T01:02:49.283908+0000","last_clean":"2024-09-18T01:02:49.283908+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:02:49.283908+0000","last_undegraded":"2024-09-18T01:02:49.283908+0000","last_fullsized":"2024-09-18T01:02:49.283908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:58:57.167086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066269099999999997,"stat_sum":{"num_bytes":38,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"26'4","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.285900+0000","last_change":"2024-09-18T01:02:43.476485+0000","last_active":"2024-09-18T01:02:49.285900+0000","last_peered":"2024-09-18T01:02:49.285900+0000","last_clean":"2024-09-18T01:02:49.285900+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:02:49.285900+0000","last_undegraded":"2024-09-18T01:02:49.285900+0000","last_fullsized":"2024-09-18T01:02:49.285900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:33.883139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054801899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'10","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.306494+0000","last_change":"2024-09-18T01:02:49.302821+0000","last_active":"2024-09-18T01:02:49.306494+0000","last_peered":"2024-09-18T01:02:49.306494+0000","last_clean":"2024-09-18T01:02:49.306494+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:02:49.306494+0000","last_undegraded":"2024-09-18T01:02:49.306494+0000","last_fullsized":"2024-09-18T01:02:49.306494+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:24:33.065619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043103,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"26'5","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.283951+0000","last_change":"2024-09-18T01:02:43.464136+0000","last_active":"2024-09-18T01:02:49.283951+0000","last_peered":"2024-09-18T01:02:49.283951+0000","last_clean":"2024-09-18T01:02:49.283951+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:02:49.283951+0000","last_undegraded":"2024-09-18T01:02:49.283951+0000","last_fullsized":"2024-09-18T01:02:49.283951+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:50.176544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061418400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"25'1","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.313176+0000","last_change":"2024-09-18T01:02:49.312596+0000","last_active":"2024-09-18T01:02:49.313176+0000","last_peered":"2024-09-18T01:02:49.313176+0000","last_clean":"2024-09-18T01:02:49.313176+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:02:49.313176+0000","last_undegraded":"2024-09-18T01:02:49.313176+0000","last_fullsized":"2024-09-18T01:02:49.313176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:37:23.039290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013236470000000001,"stat_sum":{"num_bytes":3522560,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"27'13","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.321693+0000","last_change":"2024-09-18T01:02:49.321182+0000","last_active":"2024-09-18T01:02:49.321693+0000","last_peered":"2024-09-18T01:02:49.321693+0000","last_clean":"2024-09-18T01:02:49.321693+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:02:49.321693+0000","last_undegraded":"2024-09-18T01:02:49.321693+0000","last_fullsized":"2024-09-18T01:02:49.321693+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:30.595300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011817170000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":16,"num_write":13,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"26'16","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.345972+0000","last_change":"2024-09-18T01:02:49.345428+0000","last_active":"2024-09-18T01:02:49.345972+0000","last_peered":"2024-09-18T01:02:49.345972+0000","last_clean":"2024-09-18T01:02:49.345972+0000","last_became_active":"2024-09-18T01:02:49.296123+0000","last_became_peered":"2024-09-18T01:02:49.296123+0000","last_unstale":"2024-09-18T01:02:49.345972+0000","last_undegraded":"2024-09-18T01:02:49.345972+0000","last_fullsized":"2024-09-18T01:02:49.345972+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:52:09.250851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053670300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":34,"num_write":15,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"26'3","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.283819+0000","last_change":"2024-09-18T01:02:43.463798+0000","last_active":"2024-09-18T01:02:49.283819+0000","last_peered":"2024-09-18T01:02:49.283819+0000","last_clean":"2024-09-18T01:02:49.283819+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:02:49.283819+0000","last_undegraded":"2024-09-18T01:02:49.283819+0000","last_fullsized":"2024-09-18T01:02:49.283819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:27.607818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042487699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'13","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.285721+0000","last_change":"2024-09-18T01:02:43.468742+0000","last_active":"2024-09-18T01:02:49.285721+0000","last_peered":"2024-09-18T01:02:49.285721+0000","last_clean":"2024-09-18T01:02:49.285721+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:02:49.285721+0000","last_undegraded":"2024-09-18T01:02:49.285721+0000","last_fullsized":"2024-09-18T01:02:49.285721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:49.441266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043475499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":12,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"26'7","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-18T01:02:49.356374+0000","last_change":"2024-09-18T01:02:49.356374+0000","last_active":"2024-09-18T01:02:49.356374+0000","last_peered":"2024-09-18T01:02:49.356374+0000","last_clean":"2024-09-18T01:02:49.356374+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:02:49.356374+0000","last_undegraded":"2024-09-18T01:02:49.356374+0000","last_fullsized":"2024-09-18T01:02:49.356374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:24.919497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311600000000002,"stat_sum":{"num_bytes":393216,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":12,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":16034685,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5621,"num_read_kb":5073,"num_write":693,"num_write_kb":3245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":81977,"num_keys_recovered":14,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":753402,"data_compressed":116078,"data_compressed_allocated":155648,"data_compressed_original":434176,"omap_allocated":0,"internal_metadata":0},"log_size":630,"ondisk_log_size":630,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640263,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640265,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":170964,"data_compressed":214,"data_compressed_allocated":8192,"data_compressed_original":36864,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640264,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":920894,"data_compressed":66017,"data_compressed_allocated":98304,"data_compressed_original":679936,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640263,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":331776,"data_stored":199950,"data_compressed":358,"data_compressed_allocated":8192,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640265,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":356352,"data_stored":210285,"data_compressed":425,"data_compressed_allocated":12288,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640264,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":499712,"data_stored":891295,"data_compressed":66164,"data_compressed_allocated":102400,"data_compressed_original":708608,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":192625,"data_compressed":57614,"data_compressed_allocated":65536,"data_compressed_original":135168,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":184038,"data_compressed":425,"data_compressed_allocated":12288,"data_compressed_original":81920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":91664,"data_compressed":358,"data_compressed_allocated":8192,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":222375,"data_compressed":57467,"data_compressed_allocated":61440,"data_compressed_original":106496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":62700,"data_compressed":214,"data_compressed_allocated":8192,"data_compressed_original":36864,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:02:53.669 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:53.661+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:53.670 INFO:teuthology.orchestra.run.smithi164.stdout:803 write 0x6855e00 thru 0x685e068 (0x8269 bytes) 2024-09-18T01:02:53.670 INFO:teuthology.orchestra.run.smithi164.stdout:805 write 0x69e7e00 thru 0x69f73ca (0xf5cb bytes) 2024-09-18T01:02:53.670 INFO:teuthology.orchestra.run.smithi164.stdout:806 punch from 0x1201a00 to 0x1211200, (0xf800 bytes) 2024-09-18T01:02:53.670 INFO:teuthology.orchestra.run.smithi164.stdout:807 write 0x892d200 thru 0x893030a (0x310b bytes) 2024-09-18T01:02:53.670 INFO:teuthology.orchestra.run.smithi164.stdout:809 read 0x2a73600 thru 0x2a80fae (0xd9af bytes) 2024-09-18T01:02:53.670 INFO:teuthology.orchestra.run.smithi164.stdout:812 write 0x628f800 thru 0x629091f (0x1120 bytes) 2024-09-18T01:02:53.670 INFO:teuthology.orchestra.run.smithi164.stdout:813 write 0xe451e00 thru 0xe454855 (0x2a56 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:814 punch from 0xa0afc00 to 0xa0bb200, (0xb600 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:817 read 0xbed1a00 thru 0xbedab99 (0x919a bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:819 read 0x32dda00 thru 0x32de854 (0xe55 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:820 write 0x38d6000 thru 0x38e1a7f (0xba80 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:822 read 0x1c9a200 thru 0x1ca6e37 (0xcc38 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:824 read 0xa296600 thru 0xa2a0321 (0x9d22 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:825 read 0xe20b800 thru 0xe20c00d (0x80e bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:827 read 0xd86a400 thru 0xd871e39 (0x7a3a bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:828 punch from 0xd26d600 to 0xd272400, (0x4e00 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:829 read 0xbb3fc00 thru 0xbb4ed53 (0xf154 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:831 write 0xd47dc00 thru 0xd480e36 (0x3237 bytes) 2024-09-18T01:02:53.671 INFO:teuthology.orchestra.run.smithi164.stdout:832 write 0x310f000 thru 0x3115a45 (0x6a46 bytes) 2024-09-18T01:02:53.672 INFO:teuthology.orchestra.run.smithi164.stdout:834 flatten 2024-09-18T01:02:53.672 INFO:teuthology.orchestra.run.smithi164.stdout:835 punch from 0xb7dc200 to 0xb7e8600, (0xc400 bytes) 2024-09-18T01:02:53.682 INFO:teuthology.orchestra.run.smithi164.stdout:836 trunc from 0xe454a00 to 0x215c00 2024-09-18T01:02:53.737 INFO:teuthology.orchestra.run.smithi164.stdout:837 write 0xdc8bc00 thru 0xdc977e5 (0xbbe6 bytes) 2024-09-18T01:02:53.744 INFO:teuthology.orchestra.run.smithi164.stdout:839 write 0x144fc00 thru 0x14514ec (0x18ed bytes) 2024-09-18T01:02:53.744 INFO:teuthology.orchestra.run.smithi164.stdout:845 read 0xb4b7400 thru 0xb4c2c9f (0xb8a0 bytes) 2024-09-18T01:02:53.744 INFO:teuthology.orchestra.run.smithi164.stdout:846 write 0x4b77000 thru 0x4b83668 (0xc669 bytes) 2024-09-18T01:02:53.744 INFO:teuthology.orchestra.run.smithi164.stdout:847 write 0x6147400 thru 0x614e83f (0x7440 bytes) 2024-09-18T01:02:53.744 INFO:teuthology.orchestra.run.smithi164.stdout:848 punch from 0x397ec00 to 0x3982a00, (0x3e00 bytes) 2024-09-18T01:02:53.745 INFO:teuthology.orchestra.run.smithi164.stdout:849 read 0x9c41400 thru 0x9c4e7ef (0xd3f0 bytes) 2024-09-18T01:02:53.745 INFO:teuthology.orchestra.run.smithi164.stdout:854 punch from 0x33a6e00 to 0x33b6400, (0xf600 bytes) 2024-09-18T01:02:53.745 INFO:teuthology.orchestra.run.smithi164.stdout:855 write 0xd4a0c00 thru 0xd4a893e (0x7d3f bytes) 2024-09-18T01:02:53.745 INFO:teuthology.orchestra.run.smithi164.stdout:856 read 0x7af4a00 thru 0x7aff286 (0xa887 bytes) 2024-09-18T01:02:53.745 INFO:teuthology.orchestra.run.smithi164.stdout:857 read 0xadeec00 thru 0xadf5002 (0x6403 bytes) 2024-09-18T01:02:53.748 INFO:teuthology.orchestra.run.smithi164.stdout:858 trunc from 0xdc97800 to 0xaae2c00 2024-09-18T01:02:53.762 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:53.760+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:53.770 INFO:teuthology.orchestra.run.smithi164.stdout:859 read 0x319f800 thru 0x31a82e7 (0x8ae8 bytes) 2024-09-18T01:02:53.776 INFO:teuthology.orchestra.run.smithi164.stdout:861 punch from 0xa354e00 to 0xa35c200, (0x7400 bytes) 2024-09-18T01:02:53.776 INFO:teuthology.orchestra.run.smithi164.stdout:863 write 0x7ab8600 thru 0x7ac3620 (0xb021 bytes) 2024-09-18T01:02:53.781 INFO:teuthology.orchestra.run.smithi164.stdout:865 trunc from 0xaae2c00 to 0x86d5c00 2024-09-18T01:02:53.787 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:53.788 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:53.788 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:02:53.788 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:53.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:02:53.795 INFO:teuthology.orchestra.run.smithi164.stdout:867 punch from 0x52c5e00 to 0x52d5c00, (0xfe00 bytes) 2024-09-18T01:02:53.798 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:02:53.810 INFO:teuthology.orchestra.run.smithi164.stdout:868 write 0xa359800 thru 0xa35c853 (0x3054 bytes) 2024-09-18T01:02:53.816 INFO:teuthology.orchestra.run.smithi164.stdout:869 trunc from 0xa35ca00 to 0x543d200 2024-09-18T01:02:53.833 INFO:teuthology.orchestra.run.smithi164.stdout:870 punch from 0xe1b400 to 0xe1f200, (0x3e00 bytes) 2024-09-18T01:02:53.846 INFO:teuthology.orchestra.run.smithi164.stdout:874 trunc from 0x543d200 to 0x87aa200 2024-09-18T01:02:53.866 INFO:teuthology.orchestra.run.smithi164.stdout:876 write 0xc37a200 thru 0xc386863 (0xc664 bytes) 2024-09-18T01:02:53.873 INFO:teuthology.orchestra.run.smithi164.stdout:881 read 0x1923400 thru 0x192aebe (0x7abf bytes) 2024-09-18T01:02:53.873 INFO:teuthology.orchestra.run.smithi164.stdout:883 read 0xd41a00 thru 0xd41d72 (0x373 bytes) 2024-09-18T01:02:53.873 INFO:teuthology.orchestra.run.smithi164.stdout:886 read 0x9dce200 thru 0x9dd350c (0x530d bytes) 2024-09-18T01:02:53.882 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:53.880+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:53.888 INFO:teuthology.orchestra.run.smithi164.stdout:887 write 0xe1d4c00 thru 0xe1d604a (0x144b bytes) 2024-09-18T01:02:53.889 INFO:teuthology.orchestra.run.smithi164.stdout:892 punch from 0x850be00 to 0x8512400, (0x6600 bytes) 2024-09-18T01:02:53.889 INFO:teuthology.orchestra.run.smithi164.stdout:893 read 0x8fcb400 thru 0x8fd97f2 (0xe3f3 bytes) 2024-09-18T01:02:53.890 INFO:teuthology.orchestra.run.smithi164.stdout:894 read 0xcbc1800 thru 0xcbc6895 (0x5096 bytes) 2024-09-18T01:02:53.890 INFO:teuthology.orchestra.run.smithi164.stdout:896 write 0x3307a00 thru 0x330c06b (0x466c bytes) 2024-09-18T01:02:53.890 INFO:teuthology.orchestra.run.smithi164.stdout:897 read 0xbe43000 thru 0xbe4ab4d (0x7b4e bytes) 2024-09-18T01:02:53.890 INFO:teuthology.orchestra.run.smithi164.stdout:898 clone 4 order 25 su 131072 sc 8 2024-09-18T01:02:53.920 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:53.921 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:53.938 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:53.946 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:53.946 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:53.946 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:53.946 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:53.946 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:53.956 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:53.957 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:53.957 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:02:53.964 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:53.963+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:53.970 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:53.971 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:53.988 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:54.004 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:54.065 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:54.064+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:54.100 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.100 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:54.166 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:54.164+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:54.216 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.217 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:54.266 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:54.265+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:54.335 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.335 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:54.368 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:54.366+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:54.452 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.452 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:02:54.469 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:54.467+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:54.569 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:54.568+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:54.570 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.570 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:02:54.671 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:54.669+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:54.732 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.732 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:02:54.752 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-18T01:02:54.771 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:54.770+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:54.820 INFO:teuthology.orchestra.run.smithi164.stdout:899 trunc from 0xe1d6200 to 0x8b68400 2024-09-18T01:02:54.859 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.859 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.867 INFO:teuthology.orchestra.run.smithi164.stdout:901 write 0x4cdee00 thru 0x4ceec31 (0xfe32 bytes) 2024-09-18T01:02:54.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:02:54.873 INFO:teuthology.orchestra.run.smithi164.stdout:902 write 0x57b200 thru 0x57e1d6 (0x2fd7 bytes) 2024-09-18T01:02:54.873 INFO:teuthology.orchestra.run.smithi164.stdout:904 punch from 0x885be00 to 0x8869c00, (0xde00 bytes) 2024-09-18T01:02:54.884 INFO:teuthology.orchestra.run.smithi164.stdout:906 write 0x9e5f800 thru 0x9e6461b (0x4e1c bytes) 2024-09-18T01:02:54.890 INFO:teuthology.orchestra.run.smithi164.stdout:907 read 0x9860000 thru 0x986fd49 (0xfd4a bytes) 2024-09-18T01:02:54.890 INFO:teuthology.orchestra.run.smithi164.stdout:908 punch from 0x1722200 to 0x1726a00, (0x4800 bytes) 2024-09-18T01:02:54.891 INFO:teuthology.orchestra.run.smithi164.stdout:909 clone 5 order 21 su 131072 sc 9 2024-09-18T01:02:54.912 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:54.911+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:54.973 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:54.972+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:54.978 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:54.978 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:54.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:02:55.074 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:55.073+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:55.135 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:55.135 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:55.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:02:55.175 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:55.174+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:55.252 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:55.252 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:55.261 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:02:55.276 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:55.275+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:55.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:02:55.371 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:55.371 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:55.377 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:55.376+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:55.380 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:02:55.388 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone4 intact 2024-09-18T01:02:55.478 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:55.476+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:55.483 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:55.483 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:55.492 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:02:55.580 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:55.579+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:55.637 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:55.637 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:55.646 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:02:55.690 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:55.688+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:55.732 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-18T01:02:55.751 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:55.751 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:55.760 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:02:55.801 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:55.800+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:55.882 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:55.881+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:55.903 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:55.903 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:55.912 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:02:55.983 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:55.982+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:56.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:02:56.027 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:56.027 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:56.036 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:02:56.084 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:56.083+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:56.150 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:56.150 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:56.160 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:02:56.163 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:56.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:56.202 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:56.202 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:56.204 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:56.202+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:56.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:56.239 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:56.239 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:56.260 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:56.260 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:56.275 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:56.275 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:56.275 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:56.275 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:56.275 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:56.284 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:02:56.313 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:56.312+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:56.318 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:02:56.386 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:56.385+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:56.428 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.052278662999999, 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.429 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:56.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:02:56.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:02:56.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:02:56.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.054267358000001, 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:02:56.438 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:02:56.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:02:56.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:56.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:02:56.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.046590406, 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.446 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:56.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:02:56.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 12.948732456, 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:02:56.454 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:02:56.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:02:56.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 9.0926514439999995, 2024-09-18T01:02:56.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:02:56.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 9.0922821480000007, 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.468 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:02:56.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:02:56.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:02:56.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 9.0905433030000005, 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.474 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:02:56.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 9.0904714129999995, 2024-09-18T01:02:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:02:56.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 9.0026054030000005, 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:02:56.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:02:56.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:02:56.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.9964141439999992, 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:02:56.489 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:02:56.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:02:56.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.494 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.9948844589999997, 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:02:56.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:02:56.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:02:56.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.9945047700000007, 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.501 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:02:56.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.9790063670000002, 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:02:56.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:02:56.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.9787996079999992, 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:02:56.513 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:02:56.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:02:56.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.978381851, 2024-09-18T01:02:56.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:02:56.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:02:56.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.9781726129999999, 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.524 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:02:56.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.978108679, 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:02:56.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.533 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.9773604739999993, 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:02:56.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:02:56.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.7184216689999996, 2024-09-18T01:02:56.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:02:56.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:02:56.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:02:56.544 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.7175580509999993, 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:02:56.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:02:56.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:02:56.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:02:56.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:02:56.550 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:56.552 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:56.550+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:912 read 0x4953c00 thru 0x4963246 (0xf647 bytes) 2024-09-18T01:02:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:915 write 0x4562e00 thru 0x456c889 (0x9a8a bytes) 2024-09-18T01:02:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:917 read 0x8d2dc00 thru 0x8d2de0f (0x210 bytes) 2024-09-18T01:02:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:920 trunc from 0x9e64800 to 0x32bae00 2024-09-18T01:02:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:921 read 0x368800 thru 0x3730db (0xa8dc bytes) 2024-09-18T01:02:56.553 INFO:teuthology.orchestra.run.smithi164.stdout:922 punch from 0x1191000 to 0x1193c00, (0x2c00 bytes) 2024-09-18T01:02:56.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:56.563 INFO:teuthology.orchestra.run.smithi164.stdout:924 write 0xca39800 thru 0xca3f179 (0x597a bytes) 2024-09-18T01:02:56.564 INFO:teuthology.orchestra.run.smithi164.stdout:927 punch from 0xa071000 to 0xa074800, (0x3800 bytes) 2024-09-18T01:02:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:928 read 0x4310800 thru 0x4313b5e (0x335f bytes) 2024-09-18T01:02:56.565 INFO:teuthology.orchestra.run.smithi164.stdout:930 punch from 0xa052c00 to 0xa05b200, (0x8600 bytes) 2024-09-18T01:02:56.573 INFO:teuthology.orchestra.run.smithi164.stdout:931 trunc from 0xca3f200 to 0x53c8a00 2024-09-18T01:02:56.603 INFO:teuthology.orchestra.run.smithi164.stdout:933 write 0x9aa7800 thru 0x9ab4f95 (0xd796 bytes) 2024-09-18T01:02:56.618 INFO:teuthology.orchestra.run.smithi164.stdout:934 write 0xadcc200 thru 0xadcfd79 (0x3b7a bytes) 2024-09-18T01:02:56.618 INFO:teuthology.orchestra.run.smithi164.stdout:935 read 0x1c87e00 thru 0x1c9194b (0x9b4c bytes) 2024-09-18T01:02:56.623 INFO:teuthology.orchestra.run.smithi164.stdout:936 write 0xac06a00 thru 0xac0e5e6 (0x7be7 bytes) 2024-09-18T01:02:56.623 INFO:teuthology.orchestra.run.smithi164.stdout:938 punch from 0xa543e00 to 0xa54a400, (0x6600 bytes) 2024-09-18T01:02:56.623 INFO:teuthology.orchestra.run.smithi164.stdout:939 write 0x3de0e00 thru 0x3ded730 (0xc931 bytes) 2024-09-18T01:02:56.623 INFO:teuthology.orchestra.run.smithi164.stdout:942 read 0x5329200 thru 0x5334bcf (0xb9d0 bytes) 2024-09-18T01:02:56.624 INFO:teuthology.orchestra.run.smithi164.stdout:944 clone 6 order 18 su 1024 sc 9 2024-09-18T01:02:56.653 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:56.652+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:56.671 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:56.671 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:56.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:56.753 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:56.752+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:56.786 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:56.786 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:56.786 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:56.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:56.879 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:56.878+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:56.944 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:56.944 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:130 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'88, mlcod=13'85)", 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.937376357000002, 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085906969999999996, 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004680+0000", 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004681+0000", 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5499999999999996e-07 2024-09-18T01:02:56.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004687+0000", 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7459999999999997e-06 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004697+0000", 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0513000000000001e-05 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004703+0000", 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7180000000000001e-06 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004741+0000", 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7551e-05 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.004751+0000", 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0497e-05 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013227+0000", 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084754370000000006 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013271+0000", 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4280000000000003e-05 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4177.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.936852116000001, 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080847769999999996, 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005204+0000", 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005205+0000", 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-07 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005281+0000", 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5927999999999995e-05 2024-09-18T01:02:56.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005290+0000", 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5920000000000007e-06 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005295+0000", 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.605e-06 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005342+0000", 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7327000000000003e-05 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.005364+0000", 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1583999999999999e-05 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013275+0000", 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007910594 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:27.013289+0000", 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4561e-05 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:758 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440237+0000", 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5018190340000004, 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079948129999999999, 2024-09-18T01:02:56.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 758 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440237+0000", 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440237+0000", 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440238+0000", 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.11e-07 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440239+0000", 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440242+0000", 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0469999999999998e-06 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440246+0000", 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1110000000000001e-06 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444380+0000", 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041344700000000003 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444428+0000", 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7969000000000001e-05 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444610+0000", 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018195799999999999 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447248+0000", 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026375719999999999 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448162+0000", 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091418599999999999 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448204+0000", 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1946999999999997e-05 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448232+0000", 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8102e-05 2024-09-18T01:02:56.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:760 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5018072549999992, 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01021648, 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 760 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440249+0000", 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440251+0000", 2024-09-18T01:02:56.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2979999999999999e-06 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440254+0000", 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9409999999999999e-06 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440256+0000", 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5040000000000001e-06 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444891+0000", 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046348149999999996 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.444937+0000", 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6516e-05 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445121+0000", 2024-09-18T01:02:56.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018370600000000001 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021303839999999999 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450406+0000", 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031547519999999998 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450440+0000", 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3844999999999999e-05 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450466+0000", 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5517999999999999e-05 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:761 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5017973050000002, 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010302318, 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 761 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440259+0000", 2024-09-18T01:02:56.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440261+0000", 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.469e-06 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440264+0000", 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9579999999999999e-06 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440266+0000", 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.508e-06 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445464+0000", 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051974660000000004 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445536+0000", 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1999e-05 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.445930+0000", 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039429200000000003 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447255+0000", 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013246810000000001 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450506+0000", 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032513210000000002 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450536+0000", 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0437e-05 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450561+0000", 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4983000000000001e-05 2024-09-18T01:02:56.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:762 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5017872319999999, 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010367795000000001, 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:56.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 762 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440269+0000", 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2600000000000001e-07 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440271+0000", 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440274+0000", 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.943e-06 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440276+0000", 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5210000000000001e-06 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446351+0000", 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060752339999999997 2024-09-18T01:02:56.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446400+0000", 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8437999999999999e-05 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.446610+0000", 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000209772 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.448140+0000", 2024-09-18T01:02:56.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001530214 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450587+0000", 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024474010000000001 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450614+0000", 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6769999999999999e-05 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.450637+0000", 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3110000000000001e-05 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:763 3.1b 3:daf932aa:::rbd_object_map.1103b05c2cf1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5017775709999999, 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012961914, 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 763 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440279+0000", 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9400000000000001e-07 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440280+0000", 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1829999999999999e-06 2024-09-18T01:02:56.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440283+0000", 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.931e-06 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.440286+0000", 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447287+0000", 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070015650000000004 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447380+0000", 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2727999999999997e-05 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447750+0000", 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036956400000000001 2024-09-18T01:02:56.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.452379+0000", 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046289130000000001 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453138+0000", 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075947999999999999 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453196+0000", 2024-09-18T01:02:56.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8050999999999997e-05 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453241+0000", 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4626000000000003e-05 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:829 3.1f 3:fdd092e6:::rbd_data.1103b05c2cf1.000000000000001a:head [stat,write 154624~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.487086+0000", 2024-09-18T01:02:56.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4549707749999996, 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084065200000000007, 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 829 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487086+0000", 2024-09-18T01:02:56.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487086+0000", 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487087+0000", 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7000000000000003e-07 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487102+0000", 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5900999999999999e-05 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487113+0000", 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1045e-05 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487119+0000", 2024-09-18T01:02:56.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3750000000000002e-06 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487174+0000", 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5621000000000002e-05 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487183+0000", 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0979999999999994e-06 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.492940+0000", 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057575730000000002 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493032+0000", 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1692000000000002e-05 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493457+0000", 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042539200000000002 2024-09-18T01:02:56.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.494574+0000", 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011166049999999999 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495381+0000", 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080743899999999996 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495442+0000", 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0958999999999997e-05 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495492+0000", 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9950000000000001e-05 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.993 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:1815 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.620815+0000", 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3212410509999999, 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010500943, 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1815 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:56.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620815+0000", 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620815+0000", 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620816+0000", 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.22e-07 2024-09-18T01:02:56.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620818+0000", 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9309999999999998e-06 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620830+0000", 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2006e-05 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:56.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620833+0000", 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1200000000000002e-06 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625531+0000", 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046982090000000001 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625582+0000", 2024-09-18T01:02:56.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1357999999999999e-05 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625946+0000", 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036331599999999998 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.627655+0000", 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001709148 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631208+0000", 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035527850000000001 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631269+0000", 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0989000000000002e-05 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631316+0000", 2024-09-18T01:02:56.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7759000000000001e-05 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:1816 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.620841+0000", 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3212154420000002, 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010579424, 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1816 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620841+0000", 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620841+0000", 2024-09-18T01:02:57.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620841+0000", 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.91e-07 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620843+0000", 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0109999999999999e-06 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620847+0000", 2024-09-18T01:02:57.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0620000000000002e-06 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620850+0000", 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0750000000000002e-06 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.626320+0000", 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054691109999999996 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.626397+0000", 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7340000000000002e-05 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.626685+0000", 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028821100000000002 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.629215+0000", 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025303389999999999 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631370+0000", 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021541910000000002 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631399+0000", 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8999000000000001e-05 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631420+0000", 2024-09-18T01:02:57.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1793999999999998e-05 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:1817 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.620854+0000", 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.321201989, 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010653218000000001, 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1817 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620854+0000", 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620854+0000", 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620855+0000", 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.35e-07 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620856+0000", 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6139999999999999e-06 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620859+0000", 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0759999999999999e-06 2024-09-18T01:02:57.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.620862+0000", 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9849999999999998e-06 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.627165+0000", 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063031010000000002 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.627210+0000", 2024-09-18T01:02:57.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4221999999999998e-05 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.627392+0000", 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000182726 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.629647+0000", 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022550259999999998 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631452+0000", 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001804763 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631488+0000", 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5682999999999997e-05 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631508+0000", 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9786999999999999e-05 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2319 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744289+0000", 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1977672099999999, 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079933079999999993, 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2319 2024-09-18T01:02:57.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744289+0000", 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744289+0000", 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744289+0000", 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999998e-07 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744291+0000", 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.688e-06 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744296+0000", 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744298+0000", 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7810000000000001e-06 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.745385+0000", 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010864920000000001 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.745426+0000", 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1381999999999998e-05 2024-09-18T01:02:57.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.745614+0000", 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000187808 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.749913+0000", 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042993220000000004 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752188+0000", 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022741390000000001 2024-09-18T01:02:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752242+0000", 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4279e-05 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752283+0000", 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0757000000000003e-05 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2320 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744324+0000", 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.19773247, 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080554189999999994, 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2320 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744324+0000", 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744324+0000", 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744324+0000", 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.27e-07 2024-09-18T01:02:57.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744326+0000", 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6589999999999999e-06 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744330+0000", 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2869999999999998e-06 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744334+0000", 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4769999999999999e-06 2024-09-18T01:02:57.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.745898+0000", 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015644420000000001 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.745949+0000", 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0498000000000003e-05 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.746141+0000", 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000192578 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.749917+0000", 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003775352 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752324+0000", 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024078649999999999 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752353+0000", 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8109000000000001e-05 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752379+0000", 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6825000000000002e-05 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2321 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744344+0000", 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1977128769999998, 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081848509999999999, 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2321 2024-09-18T01:02:57.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744344+0000", 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744344+0000", 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744344+0000", 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744346+0000", 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6729999999999999e-06 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744349+0000", 2024-09-18T01:02:57.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6160000000000002e-06 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744352+0000", 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.746508+0000", 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021558620000000001 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.746555+0000", 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6974000000000001e-05 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.746746+0000", 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019081999999999999 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.749920+0000", 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740150000000001 2024-09-18T01:02:57.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752437+0000", 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025167430000000001 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752489+0000", 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2278000000000003e-05 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752528+0000", 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9637e-05 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2322 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744357+0000", 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.197699187, 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082577029999999999, 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2322 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744357+0000", 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744357+0000", 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744357+0000", 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2999999999999999e-07 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744359+0000", 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.556e-06 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744362+0000", 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0850000000000001e-06 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744365+0000", 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5639999999999999e-06 2024-09-18T01:02:57.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.747025+0000", 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026605579999999999 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.747072+0000", 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6779000000000001e-05 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.747270+0000", 2024-09-18T01:02:57.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019795899999999999 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.749924+0000", 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026540750000000001 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752562+0000", 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026384210000000002 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752592+0000", 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9476e-05 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752615+0000", 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-05 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2323 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744368+0000", 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.197688592, 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009549409, 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2323 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744368+0000", 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744368+0000", 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744368+0000", 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-18T01:02:57.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744369+0000", 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3400000000000001e-06 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744373+0000", 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6799999999999999e-06 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744376+0000", 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.491e-06 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.747655+0000", 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032790749999999998 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.747716+0000", 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1523e-05 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.747967+0000", 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025067299999999998 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.750680+0000", 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027131820000000002 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.753868+0000", 2024-09-18T01:02:57.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031880049999999998 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.753897+0000", 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9026999999999998e-05 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.753917+0000", 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0203e-05 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2324 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744457+0000", 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1975997629999999, 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095654179999999991, 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2324 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744457+0000", 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744457+0000", 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744457+0000", 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744459+0000", 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744474+0000", 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4609e-05 2024-09-18T01:02:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744478+0000", 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8850000000000001e-06 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.748426+0000", 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003948582 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.748527+0000", 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010049500000000001 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.748926+0000", 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039944200000000002 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.750683+0000", 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017561950000000001 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.753956+0000", 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032733520000000002 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.753989+0000", 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3327999999999997e-05 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.754022+0000", 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2804e-05 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2325 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744488+0000", 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1975684680000001, 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096404920000000005, 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2325 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744488+0000", 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744488+0000", 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744488+0000", 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4499999999999998e-07 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744491+0000", 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2800000000000002e-06 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744495+0000", 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1439999999999996e-06 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744498+0000", 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8260000000000001e-06 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.749341+0000", 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048432179999999998 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.749395+0000", 2024-09-18T01:02:57.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4639000000000003e-05 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.749620+0000", 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000224661 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752140+0000", 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002519613 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.754059+0000", 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019194349999999999 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.754089+0000", 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0244e-05 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.754128+0000", 2024-09-18T01:02:57.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9087e-05 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2326 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744713+0000", 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1973438679999999, 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095415650000000001, 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2326 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744713+0000", 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744713+0000", 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744713+0000", 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744715+0000", 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4250000000000001e-06 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744721+0000", 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744729+0000", 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5469999999999999e-06 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.750124+0000", 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053949619999999997 2024-09-18T01:02:57.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.750182+0000", 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8909000000000001e-05 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.750412+0000", 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022926700000000001 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752388+0000", 2024-09-18T01:02:57.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019760569999999998 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.754167+0000", 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017795179999999999 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.754210+0000", 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3133999999999997e-05 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.754254+0000", 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.375e-05 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:2327 3.12 3:4f96fcac:::rbd_object_map.11031049ea29:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.744739+0000", 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.197317451, 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011916917000000001, 2024-09-18T01:02:57.061 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2327 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744739+0000", 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744739+0000", 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744739+0000", 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999999e-07 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744742+0000", 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-09-18T01:02:57.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744748+0000", 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.744756+0000", 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4149999999999999e-06 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.750723+0000", 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059665839999999996 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.750832+0000", 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109696 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.751212+0000", 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037912299999999998 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.752390+0000", 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001178512 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.756580+0000", 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041898530000000003 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.756625+0000", 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4802999999999998e-05 2024-09-18T01:02:57.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.756656+0000", 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1155999999999998e-05 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.067 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:57.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:02:57.069 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:57.067+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:57.169 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:57.168+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:57.218 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:57.218 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:57.218 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:57.218 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:57.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:02:57.273 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:57.272+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:57.337 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:57.338 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:57.347 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:02:57.371 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:57.370+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:57.424 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone5 from 0xadcfe00 (overlap 0x32bae00) to 0xa25600 2024-09-18T01:02:57.472 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:02:57.471+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:02:57.517 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:57.517 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:57.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:57.517 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:463 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256962021, 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045481109999999996, 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 463 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258237+0000", 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258238+0000", 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4900000000000005e-07 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258242+0000", 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.138e-06 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258252+0000", 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.005e-05 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258261+0000", 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8279999999999992e-06 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262585+0000", 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043244470000000004 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262670+0000", 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4451999999999997e-05 2024-09-18T01:02:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262785+0000", 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011534700000000001 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:774 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256951367999999, 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047854200000000003, 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 774 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258248+0000", 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258249+0000", 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7700000000000003e-07 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258252+0000", 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999998e-06 2024-09-18T01:02:57.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258260+0000", 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6750000000000002e-06 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258264+0000", 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2150000000000003e-06 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262838+0000", 2024-09-18T01:02:57.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045744319999999998 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.262923+0000", 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4913999999999994e-05 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263033+0000", 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110017 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:464 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256744155, 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047436980000000002, 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 464 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258455+0000", 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258456+0000", 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.78e-07 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258460+0000", 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9330000000000003e-06 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258470+0000", 2024-09-18T01:02:57.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0108e-05 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258476+0000", 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263068+0000", 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045913220000000001 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263137+0000", 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9513999999999999e-05 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263199+0000", 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1363000000000002e-05 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:775 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:57.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256710389, 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004922844, 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 775 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258489+0000", 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258490+0000", 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8800000000000001e-07 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258494+0000", 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2749999999999997e-06 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258505+0000", 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1269e-05 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258512+0000", 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.182e-06 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263231+0000", 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047188259999999997 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263299+0000", 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8009999999999994e-05 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263412+0000", 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011239399999999999 2024-09-18T01:02:57.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:465 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256338145000001, 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004784071, 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 465 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258861+0000", 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258862+0000", 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5099999999999998e-07 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258867+0000", 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2700000000000004e-06 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258879+0000", 2024-09-18T01:02:57.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1457999999999999e-05 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258886+0000", 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5220000000000001e-06 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263461+0000", 2024-09-18T01:02:57.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004574771 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263546+0000", 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4816999999999998e-05 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263645+0000", 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9382000000000005e-05 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:776 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256280437999999, 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004957576, 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:57.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 776 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258919+0000", 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4399999999999999e-07 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258924+0000", 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999998e-06 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258933+0000", 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1360000000000003e-06 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.258940+0000", 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5830000000000003e-06 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263684+0000", 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047445070000000002 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263764+0000", 2024-09-18T01:02:57.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9754999999999998e-05 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.263876+0000", 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112391 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:717 3.b 3:d07cefe4:::rbd_data.1103b05c2cf1.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.085882586, 2024-09-18T01:02:57.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046200909999999998, 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 717 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:57.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429317+0000", 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.92e-07 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429321+0000", 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8759999999999998e-06 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429331+0000", 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4700000000000008e-06 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429339+0000", 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.038e-06 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429410+0000", 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1299999999999998e-05 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429492+0000", 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2108999999999996e-05 2024-09-18T01:02:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429611+0000", 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011903100000000001 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430998+0000", 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001386649 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433835+0000", 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028371289999999999 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433892+0000", 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629e-05 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433937+0000", 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4168000000000001e-05 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:722 3.b 3:d7118d93:::rbd_data.1103b05c2cf1.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.085768499, 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046371340000000002, 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 722 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429431+0000", 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429434+0000", 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.621e-06 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429440+0000", 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9959999999999999e-06 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429445+0000", 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0420000000000002e-06 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429846+0000", 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040077599999999999 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.429929+0000", 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3208999999999996e-05 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.430033+0000", 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104388 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433745+0000", 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037116940000000002 2024-09-18T01:02:57.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.433978+0000", 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023300400000000001 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.434026+0000", 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7725999999999997e-05 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.434068+0000", 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2116000000000001e-05 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:767 3.2 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.067913731999999, 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061286680000000003, 2024-09-18T01:02:57.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 767 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:57.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447285+0000", 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447286+0000", 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9599999999999999e-07 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447289+0000", 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.853e-06 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447300+0000", 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1005999999999999e-05 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447307+0000", 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-18T01:02:57.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447346+0000", 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9787999999999999e-05 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447417+0000", 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0681999999999999e-05 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.447592+0000", 2024-09-18T01:02:57.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017477700000000001 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.449299+0000", 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017074880000000001 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453334+0000", 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040350669999999998 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453370+0000", 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5997000000000001e-05 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.453414+0000", 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3550999999999997e-05 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2380757629999994, 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:02:57.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:02:57.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2334498600000003, 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:02:57.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:02:57.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:02:57.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2333718109999996, 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:57.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:02:57.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:02:57.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:02:57.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4489.0:1143 3.16 e29/28 3:6e7c3294:::rbd_object_map.1103b05c2cf1.0000000000000004:head v 29'5, mlcod=0'0)", 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:49.301747+0000", 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.2134519909999995, 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047545599999999997, 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301747+0000", 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301747+0000", 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301748+0000", 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3200000000000005e-07 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301753+0000", 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.767e-06 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301762+0000", 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6430000000000001e-06 2024-09-18T01:02:57.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301769+0000", 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301829+0000", 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0544000000000002e-05 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.301853+0000", 2024-09-18T01:02:57.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3876e-05 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.306458+0000", 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046053070000000003 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.306502+0000", 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3284000000000001e-05 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:576 3.4 3:2761d49f:::rbd_data.1103b20ac255.0000000000000004:5 [sparse-read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.178448+0000", 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3367515760000002, 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047269770000000003, 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 576 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.178448+0000", 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.178448+0000", 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.178448+0000", 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.34e-07 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.178451+0000", 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5250000000000001e-06 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.178455+0000", 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3390000000000001e-06 2024-09-18T01:02:57.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.178457+0000", 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6620000000000001e-06 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.182979+0000", 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045214699999999997 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.183069+0000", 2024-09-18T01:02:57.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0152e-05 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.183174+0000", 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010559499999999999 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:618 3.4 3:2761d49f:::rbd_data.1103b20ac255.0000000000000004:5 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.184686+0000", 2024-09-18T01:02:57.597 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3305133910000002, 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004619794, 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 618 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184686+0000", 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184686+0000", 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184686+0000", 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000002e-07 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184688+0000", 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9750000000000001e-06 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184694+0000", 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184699+0000", 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9019999999999998e-06 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189208+0000", 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045089140000000002 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189250+0000", 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2157999999999999e-05 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189305+0000", 2024-09-18T01:02:57.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5019000000000003e-05 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:619 3.4 3:2761d49f:::rbd_data.1103b20ac255.0000000000000004:5 [sparse-read 7929856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.184726+0000", 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3304731260000002, 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048161549999999999, 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 619 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184726+0000", 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184726+0000", 2024-09-18T01:02:57.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184726+0000", 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184729+0000", 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5019999999999999e-06 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184737+0000", 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.554e-06 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184742+0000", 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4759999999999997e-06 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189351+0000", 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046091370000000001 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189439+0000", 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7870999999999999e-05 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189542+0000", 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103078 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:620 3.4 3:2761d49f:::rbd_data.1103b20ac255.0000000000000004:5 [sparse-read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.184750+0000", 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3304494199999999, 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050295139999999997, 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 620 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184750+0000", 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184750+0000", 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184750+0000", 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3300000000000002e-07 2024-09-18T01:02:57.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184752+0000", 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184758+0000", 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2129999999999996e-06 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184763+0000", 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6469999999999997e-06 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189585+0000", 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048216719999999999 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189674+0000", 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9674999999999998e-05 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189779+0000", 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104819 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:621 3.4 3:2761d49f:::rbd_data.1103b20ac255.0000000000000004:5 [sparse-read 8192000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:57.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.184770+0000", 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3304292660000003, 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052761149999999996, 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 621 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184770+0000", 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184770+0000", 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184770+0000", 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184779+0000", 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5320000000000006e-06 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184786+0000", 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1949999999999997e-06 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184792+0000", 2024-09-18T01:02:57.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9429999999999997e-06 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189843+0000", 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050515839999999996 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.189939+0000", 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5210000000000005e-05 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.190046+0000", 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107265 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:622 3.4 3:2761d49f:::rbd_data.1103b20ac255.0000000000000004:5 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.184799+0000", 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3303996299999996, 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054834879999999999, 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 622 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184799+0000", 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184799+0000", 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184800+0000", 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184802+0000", 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184808+0000", 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.184813+0000", 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9969999999999998e-06 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.190080+0000", 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052671519999999998 2024-09-18T01:02:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.190165+0000", 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.190283+0000", 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117828 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4608.0:1828 3.2 3:473f494d:::rbd_data.11031049ea29.0000000000000039:head [set-alloc-hint object_size 1048576 write_size 1048576,write 417792~40960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:53.625488+0000", 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.889710945, 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058734640000000001, 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4608", 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1828 2024-09-18T01:02:57.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625488+0000", 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625488+0000", 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:57.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625490+0000", 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6050000000000001e-06 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625534+0000", 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4412000000000001e-05 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625559+0000", 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5092e-05 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625567+0000", 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000001e-06 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625635+0000", 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8461999999999996e-05 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625783+0000", 2024-09-18T01:02:57.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147205 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.625928+0000", 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000145866 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.626711+0000", 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078235100000000003 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:57.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631277+0000", 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045664140000000004 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631319+0000", 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1604999999999999e-05 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:53.631362+0000", 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2784000000000002e-05 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:57.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:57.627 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:57.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:02:57.629 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:57.627+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:57.729 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:57.728+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:57.782 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-18T01:02:57.796 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:57.796 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:57.797 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:02:57.797 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:57.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:02:57.829 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:57.828+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:57.930 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:57.929+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:57.973 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:02:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:02:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:02:57.974 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:57.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:02:58.069 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:58.068+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:58.132 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:58.131+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:58.152 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:02:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:02:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:02:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:02:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:831 3.18 3:18ce61da:::rbd_data.1103b05c2cf1.0000000000000019:head [stat,write 393216~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:47.487246+0000", 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.662226465, 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084221350000000007, 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 831 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487246+0000", 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487246+0000", 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487248+0000", 2024-09-18T01:02:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9240000000000001e-06 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487281+0000", 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2489e-05 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487299+0000", 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7719000000000001e-05 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487306+0000", 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9300000000000003e-06 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487375+0000", 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8366000000000002e-05 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.487384+0000", 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7050000000000005e-06 2024-09-18T01:02:58.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493058+0000", 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005674775 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493134+0000", 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5598000000000004e-05 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.493292+0000", 2024-09-18T01:02:58.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015767600000000001 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495148+0000", 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018560199999999999 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495553+0000", 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040504400000000002 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495619+0000", 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6822999999999994e-05 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:47.495669+0000", 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9066000000000001e-05 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:331 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T01:02:58.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:56.522294+0000", 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.6271785510000001, 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058979549999999999, 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 331 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.522294+0000", 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.522294+0000", 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.522295+0000", 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0599999999999999e-07 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.522301+0000", 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9179999999999999e-06 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.522313+0000", 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1986e-05 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.522321+0000", 2024-09-18T01:02:58.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8679999999999999e-06 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525277+0000", 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029561019999999999 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525358+0000", 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1038999999999994e-05 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525673+0000", 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031510000000000002 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.528049+0000", 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002376155 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.528096+0000", 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6319000000000002e-05 2024-09-18T01:02:58.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.528149+0000", 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3433000000000002e-05 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.528192+0000", 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3129e-05 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:334 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:56.523836+0000", 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.625636785, 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074496019999999996, 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 334 2024-09-18T01:02:58.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.523836+0000", 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.523836+0000", 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.523837+0000", 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0160000000000001e-06 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.523842+0000", 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7990000000000001e-06 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.523856+0000", 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3733999999999999e-05 2024-09-18T01:02:58.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.523863+0000", 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5129999999999999e-06 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.527346+0000", 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034826969999999999 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.527409+0000", 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2731999999999995e-05 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.527751+0000", 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000342295 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.529881+0000", 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00213057 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531234+0000", 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013520839999999999 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531263+0000", 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9861000000000001e-05 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531286+0000", 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2300999999999999e-05 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:335 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:56.524734+0000", 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.624739272, 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067055129999999998, 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 335 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524734+0000", 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524734+0000", 2024-09-18T01:02:58.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524734+0000", 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7599999999999996e-07 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524740+0000", 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7130000000000002e-06 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524749+0000", 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3640000000000005e-06 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524755+0000", 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1730000000000003e-06 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.528469+0000", 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037147980000000001 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.528527+0000", 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7787000000000001e-05 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.528811+0000", 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000283558 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531295+0000", 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002484113 2024-09-18T01:02:58.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531342+0000", 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7494999999999998e-05 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531402+0000", 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0074000000000001e-05 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531439+0000", 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6662000000000002e-05 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:336 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:56.524764+0000", 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.6247089379999999, 2024-09-18T01:02:58.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068716489999999996, 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 336 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524764+0000", 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524764+0000", 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524764+0000", 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524767+0000", 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8499999999999998e-06 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524777+0000", 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9310000000000003e-06 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524783+0000", 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4639999999999997e-06 2024-09-18T01:02:58.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.529153+0000", 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043708439999999996 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.529216+0000", 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2384000000000004e-05 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.529553+0000", 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033697499999999999 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531300+0000", 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017475030000000001 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531582+0000", 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028182400000000002 2024-09-18T01:02:58.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531613+0000", 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0513000000000001e-05 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.531636+0000", 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3006e-05 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:337 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:56.524985+0000", 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.624488352, 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076542579999999997, 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 337 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524985+0000", 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524985+0000", 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524985+0000", 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8199999999999998e-07 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.524991+0000", 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525002+0000", 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1280999999999999e-05 2024-09-18T01:02:58.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525010+0000", 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3180000000000007e-06 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.529938+0000", 2024-09-18T01:02:58.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049274059999999996 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.530025+0000", 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7738999999999995e-05 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.530380+0000", 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035513500000000002 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.532189+0000", 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018088609999999999 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.532588+0000", 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039842700000000003 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.532617+0000", 2024-09-18T01:02:58.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8852000000000001e-05 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.532639+0000", 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2245000000000001e-05 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:338 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:56.525344+0000", 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.6241291530000002, 2024-09-18T01:02:58.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083232840000000002, 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 338 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525344+0000", 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525344+0000", 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525345+0000", 2024-09-18T01:02:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6099999999999999e-07 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525349+0000", 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6249999999999998e-06 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525361+0000", 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.15e-05 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.525368+0000", 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5460000000000003e-06 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.530724+0000", 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005355963 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.530784+0000", 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9296000000000003e-05 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.530979+0000", 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195441 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.532195+0000", 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001216186 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.533570+0000", 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001375137 2024-09-18T01:02:58.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.533626+0000", 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5937000000000003e-05 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:56.533667+0000", 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0692000000000003e-05 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:737 3.8 3:1398cb3c:::rbd_data.1103a6f5bae0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.434898+0000", 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.714574602, 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086456559999999998, 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 737 2024-09-18T01:02:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.434898+0000", 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.434898+0000", 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.434899+0000", 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.006e-06 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.434905+0000", 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9100000000000002e-06 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.434930+0000", 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4439000000000001e-05 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.434945+0000", 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4868000000000001e-05 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.435046+0000", 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101848 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.435145+0000", 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9056999999999997e-05 2024-09-18T01:02:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.435354+0000", 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020848199999999999 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.437932+0000", 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025779069999999999 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.443450+0000", 2024-09-18T01:02:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005517877 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.443515+0000", 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4968999999999994e-05 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.443544+0000", 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9292999999999999e-05 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:822 3.8 3:132e7d77:::rbd_data.1103a6f5bae0.0000000000000019:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.438478+0000", 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.71099486899999997, 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076150769999999996, 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 822 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438478+0000", 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438478+0000", 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438479+0000", 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0860000000000001e-06 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438484+0000", 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438500+0000", 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5537999999999999e-05 2024-09-18T01:02:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438507+0000", 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7989999999999994e-06 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438560+0000", 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2871000000000003e-05 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438670+0000", 2024-09-18T01:02:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110148 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.438898+0000", 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022728599999999999 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.445282+0000", 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063839769999999999 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.445997+0000", 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071522700000000003 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.446052+0000", 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911999999999999e-05 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.446093+0000", 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1352e-05 2024-09-18T01:02:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:843 3.18 3:1ec79386:::rbd_data.1103a6f5bae0.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.439739+0000", 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.70973386400000005, 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011504841, 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 843 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439739+0000", 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439739+0000", 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439740+0000", 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0200000000000001e-07 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439745+0000", 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-06 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439753+0000", 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9920000000000007e-06 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439760+0000", 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6529999999999997e-06 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439834+0000", 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4381999999999994e-05 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439908+0000", 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4564e-05 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440091+0000", 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018205599999999999 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.446016+0000", 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059252799999999998 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.451166+0000", 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005150165 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.451202+0000", 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6418999999999998e-05 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.451244+0000", 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1458000000000002e-05 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4615.0:842 3.17 e36/28 3:ec5f7238:::rbd_data.1103a6f5bae0.000000000000000d:head v 36'46, mlcod=34'44)", 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.439783+0000", 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.70968971800000002, 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055394670000000002, 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439783+0000", 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439783+0000", 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439784+0000", 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999997e-07 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439791+0000", 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.477e-06 2024-09-18T01:02:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439801+0000", 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0806e-05 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439808+0000", 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4049999999999997e-06 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439864+0000", 2024-09-18T01:02:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6694999999999998e-05 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.439876+0000", 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1922e-05 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.445281+0000", 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054047119999999999 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.445323+0000", 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1539999999999999e-05 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:844 3.18 3:18113fa0:::rbd_data.1103a6f5bae0.000000000000001a:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.440715+0000", 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.70875800200000005, 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010597245999999999, 2024-09-18T01:02:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 844 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440715+0000", 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440715+0000", 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440716+0000", 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0999999999999998e-07 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440719+0000", 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7869999999999999e-06 2024-09-18T01:02:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440730+0000", 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0181e-05 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440734+0000", 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9350000000000002e-06 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440783+0000", 2024-09-18T01:02:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8430999999999998e-05 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440834+0000", 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0943999999999999e-05 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.440925+0000", 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1478999999999996e-05 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.446056+0000", 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051305789999999997 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.451250+0000", 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051941449999999998 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.451277+0000", 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6936000000000001e-05 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.451312+0000", 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5119e-05 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4615.0:846 3.9 e36/20 3:920ff767:::rbd_data.1103a6f5bae0.000000000000004c:head v 36'11, mlcod=35'8)", 2024-09-18T01:02:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.441296+0000", 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.70817709500000003, 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096035909999999999, 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441296+0000", 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441296+0000", 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441296+0000", 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.22e-07 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441301+0000", 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0459999999999996e-06 2024-09-18T01:02:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441314+0000", 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.288e-05 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441321+0000", 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441398+0000", 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7538000000000002e-05 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.441422+0000", 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.338e-05 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.450807+0000", 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093852399999999996 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.450899+0000", 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2281999999999995e-05 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4615.0:861 3.0 e36/20 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:head v 36'25, mlcod=36'20)", 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.447089+0000", 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.70238344900000005, 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065828509999999998, 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.447089+0000", 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.447089+0000", 2024-09-18T01:02:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.447090+0000", 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5199999999999996e-07 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.447105+0000", 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.505e-05 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.447118+0000", 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3016e-05 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.447126+0000", 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4089999999999997e-06 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.448084+0000", 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00095855500000000004 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.448103+0000", 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8853999999999999e-05 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.453652+0000", 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055493039999999997 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.453672+0000", 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0010999999999999e-05 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:995 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.461005+0000", 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.68846801400000002, 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057878189999999996, 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 995 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461005+0000", 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461005+0000", 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461005+0000", 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461008+0000", 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9849999999999998e-06 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461016+0000", 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1609999999999997e-06 2024-09-18T01:02:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461022+0000", 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7329999999999997e-06 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.464446+0000", 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034234510000000001 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.464528+0000", 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2600000000000002e-05 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.464876+0000", 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034755300000000001 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.465929+0000", 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001052765 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466719+0000", 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079058000000000004 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466754+0000", 2024-09-18T01:02:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4885999999999998e-05 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466793+0000", 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8602000000000001e-05 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:996 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.461030+0000", 2024-09-18T01:02:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.68844323500000004, 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085404279999999992, 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 996 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461030+0000", 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461030+0000", 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461030+0000", 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461036+0000", 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2369999999999997e-06 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461044+0000", 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3019999999999997e-06 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461049+0000", 2024-09-18T01:02:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6380000000000001e-06 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.465132+0000", 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040823500000000002 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.465181+0000", 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9067000000000003e-05 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.465362+0000", 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018154 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466835+0000", 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014728759999999999 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469488+0000", 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002652593 2024-09-18T01:02:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469539+0000", 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1075999999999997e-05 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469570+0000", 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1270999999999999e-05 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:997 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.461057+0000", 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.68841611599999997, 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085923990000000006, 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 997 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461057+0000", 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461057+0000", 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461057+0000", 2024-09-18T01:02:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8699999999999995e-07 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461060+0000", 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.604e-06 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461067+0000", 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9129999999999997e-06 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.461072+0000", 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4870000000000002e-06 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.465960+0000", 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048872550000000001 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466004+0000", 2024-09-18T01:02:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4665e-05 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466328+0000", 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032422499999999998 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.468528+0000", 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021997230000000002 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469578+0000", 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010497410000000001 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469628+0000", 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9740999999999997e-05 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469649+0000", 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1557999999999999e-05 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:998 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.462056+0000", 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.68741659399999999, 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076382660000000003, 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 998 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462056+0000", 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462056+0000", 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462057+0000", 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.17e-06 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462064+0000", 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9290000000000001e-06 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462076+0000", 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1715e-05 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462087+0000", 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0852999999999999e-05 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466857+0000", 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047702539999999998 2024-09-18T01:02:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.466925+0000", 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8057000000000002e-05 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.467287+0000", 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000361944 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.468585+0000", 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012980929999999999 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469654+0000", 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001068844 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469677+0000", 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3047000000000001e-05 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469695+0000", 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.736e-05 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:1000 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.462652+0000", 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.68682138500000001, 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072243330000000003, 2024-09-18T01:02:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1000 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462652+0000", 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462652+0000", 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462652+0000", 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9199999999999999e-07 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462657+0000", 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462669+0000", 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1892999999999999e-05 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.462677+0000", 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.467672+0000", 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004995457 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.467752+0000", 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9437999999999994e-05 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.468199+0000", 2024-09-18T01:02:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044769500000000002 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469706+0000", 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001506783 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469752+0000", 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000002e-05 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469830+0000", 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7453999999999994e-05 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469876+0000", 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6035000000000002e-05 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4615.0:1002 3.8 3:14af5599:::rbd_object_map.1103a6f5bae0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:02:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:57.463165+0000", 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.68630740700000004, 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007202183, 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4615", 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/208715925", 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1002 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.463165+0000", 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.463165+0000", 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.463166+0000", 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3900000000000003e-07 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.463171+0000", 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8339999999999998e-06 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.463181+0000", 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5710000000000004e-06 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.463188+0000", 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9190000000000002e-06 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.468629+0000", 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054417500000000004 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.468719+0000", 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9881000000000002e-05 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469093+0000", 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037315800000000001 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.469891+0000", 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079836499999999999 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.470268+0000", 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037733999999999999 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.470327+0000", 2024-09-18T01:02:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8289999999999999e-05 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:57.470368+0000", 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1136000000000001e-05 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:02:58.274 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:02:58.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:02:58.276 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:58.274+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:58.277 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:02:58.299 INFO:teuthology.orchestra.run.smithi164.stdout:948 trunc from 0xadcfe00 to 0x2063c00 2024-09-18T01:02:58.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:02:58.376 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:58.375+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:58.398 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:58.399 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:58.399 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:02:58.399 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:58.407 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:02:58.469 INFO:teuthology.orchestra.run.smithi164.stdout:950 trunc from 0x2063c00 to 0xa989a00 2024-09-18T01:02:58.479 INFO:teuthology.orchestra.run.smithi164.stdout:955 read 0x6046c00 thru 0x604e823 (0x7c24 bytes) 2024-09-18T01:02:58.491 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:58.490+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:58.492 INFO:teuthology.orchestra.run.smithi164.stdout:956 trunc from 0xa989a00 to 0x54b4800 2024-09-18T01:02:58.500 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:02:58.500 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:02:58.516 INFO:teuthology.orchestra.run.smithi164.stdout:958 trunc from 0x54b4800 to 0x8e08a00 2024-09-18T01:02:58.530 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:02:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:02:58.550 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:02:58.551 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:02:58.561 INFO:teuthology.orchestra.run.smithi164.stdout:959 read 0x1e31e00 thru 0x1e3c6a4 (0xa8a5 bytes) 2024-09-18T01:02:58.565 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:02:58.565 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:02:58.565 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:02:58.566 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:02:58.566 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:02:58.567 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:02:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:02:58.575 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:02:58.575 INFO:teuthology.orchestra.run.smithi164.stdout:960 trunc from 0x8e08a00 to 0xe02f200 2024-09-18T01:02:58.585 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:02:58.585 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:02:58.604 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:02:58.604 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:02:58.611 INFO:teuthology.orchestra.run.smithi164.stdout:961 punch from 0x272d600 to 0x2735400, (0x7e00 bytes) 2024-09-18T01:02:58.611 INFO:teuthology.orchestra.run.smithi164.stdout:962 punch from 0x9d06400 to 0x9d13a00, (0xd600 bytes) 2024-09-18T01:02:58.611 INFO:teuthology.orchestra.run.smithi164.stdout:963 write 0x4286800 thru 0x428d096 (0x6897 bytes) 2024-09-18T01:02:58.617 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:58.616+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:58.622 INFO:teuthology.orchestra.run.smithi164.stdout:967 trunc from 0xe02f200 to 0x49e0600 2024-09-18T01:02:58.670 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:02:58.670 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:02:58.670 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T01:02:58.670 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-18T01:02:58.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T01:02:58.678 INFO:teuthology.orchestra.run.smithi164.stdout:968 trunc from 0x49e0600 to 0x17f600 2024-09-18T01:02:58.714 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:58.713+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:58.734 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:58.734 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:58.747 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:02:58.780 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:58.778+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:58.832 INFO:teuthology.orchestra.run.smithi164.stdout:970 read 0x158800 thru 0x158d1a (0x51b bytes) 2024-09-18T01:02:58.839 INFO:teuthology.orchestra.run.smithi164.stdout:972 trunc from 0x17f600 to 0x2738e00 2024-09-18T01:02:58.865 INFO:teuthology.orchestra.run.smithi164.stdout:973 write 0x366fc00 thru 0x367dcb3 (0xe0b4 bytes) 2024-09-18T01:02:58.874 INFO:teuthology.orchestra.run.smithi164.stdout:975 trunc from 0x367de00 to 0x6790000 2024-09-18T01:02:58.880 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:58.879+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:58.881 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:58.881 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:58.891 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:02:58.910 INFO:teuthology.orchestra.run.smithi164.stdout:977 write 0xc813c00 thru 0xc817d6d (0x416e bytes) 2024-09-18T01:02:58.926 INFO:teuthology.orchestra.run.smithi164.stdout:978 trunc from 0xc817e00 to 0x2bdbc00 2024-09-18T01:02:58.958 INFO:teuthology.orchestra.run.smithi164.stdout:980 read 0x1dee00 thru 0x1edd89 (0xef8a bytes) 2024-09-18T01:02:58.981 INFO:teuthology.orchestra.run.smithi164.stdout:981 write 0x7d7d000 thru 0x7d847f6 (0x77f7 bytes) 2024-09-18T01:02:58.981 INFO:teuthology.orchestra.run.smithi164.stdout:982 write 0x42f6000 thru 0x4300397 (0xa398 bytes) 2024-09-18T01:02:58.981 INFO:teuthology.orchestra.run.smithi164.stdout:985 read 0x3f08a00 thru 0x3f17607 (0xec08 bytes) 2024-09-18T01:02:58.981 INFO:teuthology.orchestra.run.smithi164.stdout:986 write 0x3738c00 thru 0x37470b6 (0xe4b7 bytes) 2024-09-18T01:02:58.982 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:02:58.979+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:02:58.982 INFO:teuthology.orchestra.run.smithi164.stdout:988 punch from 0x3c5a200 to 0x3c68200, (0xe000 bytes) 2024-09-18T01:02:58.991 INFO:teuthology.orchestra.run.smithi164.stdout:989 trunc from 0x7d84800 to 0x20e5a00 2024-09-18T01:02:59.004 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:59.004 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:59.014 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:02:59.059 INFO:teuthology.orchestra.run.smithi164.stdout:990 trunc from 0x20e5a00 to 0xc8c8a00 2024-09-18T01:02:59.078 INFO:teuthology.orchestra.run.smithi164.stdout:994 trunc from 0xc8c8a00 to 0x94da400 2024-09-18T01:02:59.090 INFO:teuthology.orchestra.run.smithi164.stdout:995 read 0x4a5b800 thru 0x4a5bd66 (0x567 bytes) 2024-09-18T01:02:59.096 INFO:teuthology.orchestra.run.smithi164.stdout:997 write 0x2fd8e00 thru 0x2fe17c0 (0x89c1 bytes) 2024-09-18T01:02:59.096 INFO:teuthology.orchestra.run.smithi164.stdout:998 read 0xf01000 thru 0xf08d91 (0x7d92 bytes) 2024-09-18T01:02:59.097 INFO:teuthology.orchestra.run.smithi164.stdout:999 punch from 0x2978000 to 0x2982c00, (0xac00 bytes) 2024-09-18T01:02:59.098 INFO:teuthology.orchestra.run.smithi164.stdout:1002 trunc from 0x94da400 to 0x9e23c00 2024-09-18T01:02:59.098 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:02:59.097+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:02:59.126 INFO:teuthology.orchestra.run.smithi164.stdout:1003 punch from 0x2929600 to 0x292a400, (0xe00 bytes) 2024-09-18T01:02:59.133 INFO:teuthology.orchestra.run.smithi164.stdout:1004 read 0xaa9e00 thru 0xaac692 (0x2893 bytes) 2024-09-18T01:02:59.134 INFO:teuthology.orchestra.run.smithi164.stdout:1007 read 0x2572a00 thru 0x257cbe6 (0xa1e7 bytes) 2024-09-18T01:02:59.134 INFO:teuthology.orchestra.run.smithi164.stdout:1008 write 0x198a00 thru 0x1a13d4 (0x89d5 bytes) 2024-09-18T01:02:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:1010 read 0x4170400 thru 0x4173d90 (0x3991 bytes) 2024-09-18T01:02:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:1011 write 0x42a4a00 thru 0x42b256a (0xdb6b bytes) 2024-09-18T01:02:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:1012 write 0x3ef2600 thru 0x3ef679e (0x419f bytes) 2024-09-18T01:02:59.135 INFO:teuthology.orchestra.run.smithi164.stdout:1014 read 0x2489600 thru 0x248b631 (0x2032 bytes) 2024-09-18T01:02:59.167 INFO:teuthology.orchestra.run.smithi164.stdout:1016 write 0xa9aa800 thru 0xa9b7801 (0xd002 bytes) 2024-09-18T01:02:59.168 INFO:teuthology.orchestra.run.smithi164.stdout:1018 read 0x88d7a00 thru 0x88e3504 (0xbb05 bytes) 2024-09-18T01:02:59.168 INFO:teuthology.orchestra.run.smithi164.stdout:1021 read 0xa790400 thru 0xa799be5 (0x97e6 bytes) 2024-09-18T01:02:59.171 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:59.171 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:59.181 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:02:59.182 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:59.181+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:59.190 INFO:teuthology.orchestra.run.smithi164.stdout:1022 write 0xc08b600 thru 0xc097e7f (0xc880 bytes) 2024-09-18T01:02:59.220 INFO:teuthology.orchestra.run.smithi164.stdout:1025 write 0xd3d1200 thru 0xd3d7f8b (0x6d8c bytes) 2024-09-18T01:02:59.220 INFO:teuthology.orchestra.run.smithi164.stdout:1027 punch from 0x9740a00 to 0x9740e00, (0x400 bytes) 2024-09-18T01:02:59.225 INFO:teuthology.orchestra.run.smithi164.stdout:1028 read 0xd306e00 thru 0xd31092e (0x9b2f bytes) 2024-09-18T01:02:59.225 INFO:teuthology.orchestra.run.smithi164.stdout:1030 write 0x9689a00 thru 0x968bde3 (0x23e4 bytes) 2024-09-18T01:02:59.225 INFO:teuthology.orchestra.run.smithi164.stdout:1032 write 0x8cd5400 thru 0x8cd77ad (0x23ae bytes) 2024-09-18T01:02:59.229 INFO:teuthology.orchestra.run.smithi164.stdout:1034 trunc from 0xd3d8000 to 0x9950a00 2024-09-18T01:02:59.281 INFO:teuthology.orchestra.run.smithi164.stdout:1035 write 0xd5a400 thru 0xd5eec8 (0x4ac9 bytes) 2024-09-18T01:02:59.283 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:59.282+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:59.290 INFO:teuthology.orchestra.run.smithi164.stdout:1037 write 0x675400 thru 0x67b3bd (0x5fbe bytes) 2024-09-18T01:02:59.291 INFO:teuthology.orchestra.run.smithi164.stdout:1038 write 0x7f31400 thru 0x7f3700f (0x5c10 bytes) 2024-09-18T01:02:59.300 INFO:teuthology.orchestra.run.smithi164.stdout:1041 trunc from 0x9950a00 to 0x16c6800 2024-09-18T01:02:59.340 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T01:02:59.344 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:59.345 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:02:59.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:02:59.384 INFO:teuthology.orchestra.run.smithi164.stdout:1042 read 0xdf9800 thru 0xe025f6 (0x8df7 bytes) 2024-09-18T01:02:59.409 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:59.407+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:59.417 INFO:teuthology.orchestra.run.smithi164.stdout:1043 write 0xa282600 thru 0xa2866f5 (0x40f6 bytes) 2024-09-18T01:02:59.427 INFO:teuthology.orchestra.run.smithi164.stdout:1045 read 0x7bb0c00 thru 0x7bb4b87 (0x3f88 bytes) 2024-09-18T01:02:59.427 INFO:teuthology.orchestra.run.smithi164.stdout:1047 read 0x8dafa00 thru 0x8dbcef8 (0xd4f9 bytes) 2024-09-18T01:02:59.427 INFO:teuthology.orchestra.run.smithi164.stdout:1048 write 0x1537a00 thru 0x1542902 (0xaf03 bytes) 2024-09-18T01:02:59.427 INFO:teuthology.orchestra.run.smithi164.stdout:1050 write 0x1901400 thru 0x1907bed (0x67ee bytes) 2024-09-18T01:02:59.427 INFO:teuthology.orchestra.run.smithi164.stdout:1051 read 0x3adf600 thru 0x3aea7ad (0xb1ae bytes) 2024-09-18T01:02:59.468 INFO:teuthology.orchestra.run.smithi164.stdout:1052 write 0xebe9200 thru 0xebe98ec (0x6ed bytes) 2024-09-18T01:02:59.471 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:59.471 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:59.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:02:59.486 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:59.484+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:59.486 INFO:teuthology.orchestra.run.smithi164.stdout:1053 trunc from 0xebe9a00 to 0x17d3000 2024-09-18T01:02:59.523 INFO:teuthology.orchestra.run.smithi164.stdout:1055 read 0xb50200 thru 0xb5c7f4 (0xc5f5 bytes) 2024-09-18T01:02:59.544 INFO:teuthology.orchestra.run.smithi164.stdout:1056 write 0x95b3c00 thru 0x95bd3e3 (0x97e4 bytes) 2024-09-18T01:02:59.545 INFO:teuthology.orchestra.run.smithi164.stdout:1057 read 0x22a9c00 thru 0x22ae6c0 (0x4ac1 bytes) 2024-09-18T01:02:59.552 INFO:teuthology.orchestra.run.smithi164.stdout:1058 trunc from 0x95bd400 to 0x2fd4c00 2024-09-18T01:02:59.579 INFO:teuthology.orchestra.run.smithi164.stdout:1059 read 0xefbe00 thru 0xf06a4e (0xac4f bytes) 2024-09-18T01:02:59.585 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:59.584+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:02:59.593 INFO:teuthology.orchestra.run.smithi164.stdout:1062 write 0x5e88000 thru 0x5e8d981 (0x5982 bytes) 2024-09-18T01:02:59.594 INFO:teuthology.orchestra.run.smithi164.stdout:1063 write 0x3b2bc00 thru 0x3b3880c (0xcc0d bytes) 2024-09-18T01:02:59.595 INFO:teuthology.orchestra.run.smithi164.stdout:1066 trunc from 0x5e8da00 to 0x6790200 2024-09-18T01:02:59.601 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:59.601 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:59.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:02:59.628 INFO:teuthology.orchestra.run.smithi164.stdout:1067 write 0x22e4a00 thru 0x22f2307 (0xd908 bytes) 2024-09-18T01:02:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:1068 write 0xa9e6e00 thru 0xa9ed8b0 (0x6ab1 bytes) 2024-09-18T01:02:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:1069 read 0x2db7c00 thru 0x2dc3732 (0xbb33 bytes) 2024-09-18T01:02:59.659 INFO:teuthology.orchestra.run.smithi164.stdout:1071 read 0x1b2600 thru 0x1bb2d0 (0x8cd1 bytes) 2024-09-18T01:02:59.667 INFO:teuthology.orchestra.run.smithi164.stdout:1072 trunc from 0xa9eda00 to 0x4d73e00 2024-09-18T01:02:59.686 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:59.685+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:59.711 INFO:teuthology.orchestra.run.smithi164.stdout:1073 write 0x5e8fc00 thru 0x5e97a30 (0x7e31 bytes) 2024-09-18T01:02:59.712 INFO:teuthology.orchestra.run.smithi164.stdout:1074 write 0x115b400 thru 0x11677e9 (0xc3ea bytes) 2024-09-18T01:02:59.712 INFO:teuthology.orchestra.run.smithi164.stdout:1075 write 0x1b71c00 thru 0x1b7a662 (0x8a63 bytes) 2024-09-18T01:02:59.751 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:59.751 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:59.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:02:59.761 INFO:teuthology.orchestra.run.smithi164.stdout:1082 write 0xcdaf800 thru 0xcdb7029 (0x782a bytes) 2024-09-18T01:02:59.762 INFO:teuthology.orchestra.run.smithi164.stdout:1085 write 0x2bbe000 thru 0x2bca316 (0xc317 bytes) 2024-09-18T01:02:59.763 INFO:teuthology.orchestra.run.smithi164.stdout:1086 write 0xaa9ac00 thru 0xaaa8078 (0xd479 bytes) 2024-09-18T01:02:59.788 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:02:59.787+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:02:59.806 INFO:teuthology.orchestra.run.smithi164.stdout:1087 write 0xec27000 thru 0xec30800 (0x9801 bytes) 2024-09-18T01:02:59.806 INFO:teuthology.orchestra.run.smithi164.stdout:1095 punch from 0x8ab0800 to 0x8ab3800, (0x3000 bytes) 2024-09-18T01:02:59.808 INFO:teuthology.orchestra.run.smithi164.stdout:1096 read 0xc648c00 thru 0xc651059 (0x845a bytes) 2024-09-18T01:02:59.808 INFO:teuthology.orchestra.run.smithi164.stdout:1097 read 0xcca9c00 thru 0xccab059 (0x145a bytes) 2024-09-18T01:02:59.808 INFO:teuthology.orchestra.run.smithi164.stdout:1098 read 0x4736000 thru 0x4741d08 (0xbd09 bytes) 2024-09-18T01:02:59.808 INFO:teuthology.orchestra.run.smithi164.stdout:1099 read 0x5271200 thru 0x5274f4b (0x3d4c bytes) 2024-09-18T01:02:59.808 INFO:teuthology.orchestra.run.smithi164.stdout:1101 punch from 0xd996c00 to 0xd99f600, (0x8a00 bytes) 2024-09-18T01:02:59.809 INFO:teuthology.orchestra.run.smithi164.stdout:1102 write 0x8869000 thru 0x8878fb3 (0xffb4 bytes) 2024-09-18T01:02:59.809 INFO:teuthology.orchestra.run.smithi164.stdout:1103 write 0x452c00 thru 0x45858f (0x5990 bytes) 2024-09-18T01:02:59.809 INFO:teuthology.orchestra.run.smithi164.stdout:1104 read 0x594b600 thru 0x5959df6 (0xe7f7 bytes) 2024-09-18T01:02:59.811 INFO:teuthology.orchestra.run.smithi164.stdout:1107 read 0xc92e600 thru 0xc93087f (0x2280 bytes) 2024-09-18T01:02:59.817 INFO:teuthology.orchestra.run.smithi164.stdout:1110 trunc from 0xec30a00 to 0x7288200 2024-09-18T01:02:59.889 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:02:59.888+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:02:59.895 INFO:teuthology.orchestra.run.smithi164.stdout:1111 punch from 0x1e1da00 to 0x1e29600, (0xbc00 bytes) 2024-09-18T01:02:59.900 INFO:teuthology.orchestra.run.smithi164.stdout:1112 read 0x6253c00 thru 0x6260d64 (0xd165 bytes) 2024-09-18T01:02:59.905 INFO:teuthology.orchestra.run.smithi164.stdout:1113 trunc from 0x7288200 to 0x2f5f600 2024-09-18T01:02:59.940 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:02:59.940 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:02:59.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:02:59.956 INFO:teuthology.orchestra.run.smithi164.stdout:1115 write 0xb1a4c00 thru 0xb1ae3b5 (0x97b6 bytes) 2024-09-18T01:02:59.962 INFO:teuthology.orchestra.run.smithi164.stdout:1116 write 0x9ff200 thru 0xa027ab (0x35ac bytes) 2024-09-18T01:02:59.962 INFO:teuthology.orchestra.run.smithi164.stdout:1123 read 0x430e00 thru 0x43db48 (0xcd49 bytes) 2024-09-18T01:02:59.962 INFO:teuthology.orchestra.run.smithi164.stdout:1125 punch from 0x5ed7c00 to 0x5ee7600, (0xfa00 bytes) 2024-09-18T01:02:59.994 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:02:59.993+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:00.000 INFO:teuthology.orchestra.run.smithi164.stdout:1127 write 0xe4f0a00 thru 0xe4fc80e (0xbe0f bytes) 2024-09-18T01:03:00.001 INFO:teuthology.orchestra.run.smithi164.stdout:1128 read 0x84a9400 thru 0x84b6d43 (0xd944 bytes) 2024-09-18T01:03:00.001 INFO:teuthology.orchestra.run.smithi164.stdout:1129 write 0x282dc00 thru 0x28369db (0x8ddc bytes) 2024-09-18T01:03:00.001 INFO:teuthology.orchestra.run.smithi164.stdout:1130 write 0xb49c400 thru 0xb4ac2ed (0xfeee bytes) 2024-09-18T01:03:00.001 INFO:teuthology.orchestra.run.smithi164.stdout:1132 read 0x70f6000 thru 0x70fbe67 (0x5e68 bytes) 2024-09-18T01:03:00.001 INFO:teuthology.orchestra.run.smithi164.stdout:1134 write 0x69cf600 thru 0x69d9e85 (0xa886 bytes) 2024-09-18T01:03:00.001 INFO:teuthology.orchestra.run.smithi164.stdout:1135 read 0xbbd9800 thru 0xbbdf5b8 (0x5db9 bytes) 2024-09-18T01:03:00.010 INFO:teuthology.orchestra.run.smithi164.stdout:1138 trunc from 0xe4fca00 to 0x67f2e00 2024-09-18T01:03:00.083 INFO:teuthology.orchestra.run.smithi164.stdout:1139 write 0x34da600 thru 0x34dce4a (0x284b bytes) 2024-09-18T01:03:00.084 INFO:teuthology.orchestra.run.smithi164.stdout:1140 read 0x1417a00 thru 0x141f67f (0x7c80 bytes) 2024-09-18T01:03:00.084 INFO:teuthology.orchestra.run.smithi164.stdout:1141 punch from 0x424ae00 to 0x424d600, (0x2800 bytes) 2024-09-18T01:03:00.086 INFO:teuthology.orchestra.run.smithi164.stdout:1145 trunc from 0x67f2e00 to 0x718de00 2024-09-18T01:03:00.091 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:00.090+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout:1148 read 0x301f200 thru 0x30287aa (0x95ab bytes) 2024-09-18T01:03:00.119 INFO:teuthology.orchestra.run.smithi164.stdout:1151 punch from 0x1728200 to 0x172c400, (0x4200 bytes) 2024-09-18T01:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout:1153 write 0x2d65600 thru 0x2d67db3 (0x27b4 bytes) 2024-09-18T01:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout:1156 read 0x40f8200 thru 0x4107725 (0xf526 bytes) 2024-09-18T01:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout:1157 read 0x5716600 thru 0x571d275 (0x6c76 bytes) 2024-09-18T01:03:00.121 INFO:teuthology.orchestra.run.smithi164.stdout:1158 punch from 0x6236e00 to 0x623c200, (0x5400 bytes) 2024-09-18T01:03:00.123 INFO:teuthology.orchestra.run.smithi164.stdout:1159 trunc from 0x718de00 to 0x5d6aa00 2024-09-18T01:03:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:00.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:00.191 INFO:teuthology.orchestra.run.smithi164.stdout:1161 write 0xdedc200 thru 0xdee07f7 (0x45f8 bytes) 2024-09-18T01:03:00.192 INFO:teuthology.orchestra.run.smithi164.stdout:1164 punch from 0x43fbc00 to 0x4409c00, (0xe000 bytes) 2024-09-18T01:03:00.192 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:00.191+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout:1165 read 0x2fedc00 thru 0x2ff1ac4 (0x3ec5 bytes) 2024-09-18T01:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout:1166 read 0x6811e00 thru 0x681cabe (0xacbf bytes) 2024-09-18T01:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout:1168 write 0xddbae00 thru 0xddc0a79 (0x5c7a bytes) 2024-09-18T01:03:00.193 INFO:teuthology.orchestra.run.smithi164.stdout:1170 punch from 0xf7000 to 0xfa800, (0x3800 bytes) 2024-09-18T01:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout:1172 read 0xb1ae200 thru 0xb1b6741 (0x8542 bytes) 2024-09-18T01:03:00.194 INFO:teuthology.orchestra.run.smithi164.stdout:1175 punch from 0x9b98c00 to 0x9ba8000, (0xf400 bytes) 2024-09-18T01:03:00.195 INFO:teuthology.orchestra.run.smithi164.stdout:1178 read 0x2ad6a00 thru 0x2ad7a02 (0x1003 bytes) 2024-09-18T01:03:00.200 INFO:teuthology.orchestra.run.smithi164.stdout:1179 trunc from 0xdee0800 to 0x904d000 2024-09-18T01:03:00.261 INFO:teuthology.orchestra.run.smithi164.stdout:1180 read 0x7c69a00 thru 0x7c78446 (0xea47 bytes) 2024-09-18T01:03:00.262 INFO:teuthology.orchestra.run.smithi164.stdout:1182 write 0x1c50800 thru 0x1c5b450 (0xac51 bytes) 2024-09-18T01:03:00.276 INFO:teuthology.orchestra.run.smithi164.stdout:1183 write 0xa592400 thru 0xa5a0a79 (0xe67a bytes) 2024-09-18T01:03:00.277 INFO:teuthology.orchestra.run.smithi164.stdout:1184 read 0x1e3c000 thru 0x1e48287 (0xc288 bytes) 2024-09-18T01:03:00.278 INFO:teuthology.orchestra.run.smithi164.stdout:1185 write 0x86a2c00 thru 0x86a36a8 (0xaa9 bytes) 2024-09-18T01:03:00.278 INFO:teuthology.orchestra.run.smithi164.stdout:1186 punch from 0x8376c00 to 0x8381000, (0xa400 bytes) 2024-09-18T01:03:00.279 INFO:teuthology.orchestra.run.smithi164.stdout:1191 punch from 0x1e19000 to 0x1e26c00, (0xdc00 bytes) 2024-09-18T01:03:00.280 INFO:teuthology.orchestra.run.smithi164.stdout:1193 write 0x4acd600 thru 0x4ad15fa (0x3ffb bytes) 2024-09-18T01:03:00.280 INFO:teuthology.orchestra.run.smithi164.stdout:1194 write 0x4b5e00 thru 0x4c0c8d (0xae8e bytes) 2024-09-18T01:03:00.280 INFO:teuthology.orchestra.run.smithi164.stdout:1195 read 0x73e4c00 thru 0x73f462d (0xfa2e bytes) 2024-09-18T01:03:00.292 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:00.291+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:00.325 INFO:teuthology.orchestra.run.smithi164.stdout:1196 write 0xae0c800 thru 0xae1bb33 (0xf334 bytes) 2024-09-18T01:03:00.333 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:00.333 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:00.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:00.356 INFO:teuthology.orchestra.run.smithi164.stdout:1197 write 0xe5d4c00 thru 0xe5de5d7 (0x99d8 bytes) 2024-09-18T01:03:00.356 INFO:teuthology.orchestra.run.smithi164.stdout:1199 write 0x470be00 thru 0x470ec49 (0x2e4a bytes) 2024-09-18T01:03:00.356 INFO:teuthology.orchestra.run.smithi164.stdout:1200 punch from 0xc95f600 to 0xc967400, (0x7e00 bytes) 2024-09-18T01:03:00.357 INFO:teuthology.orchestra.run.smithi164.stdout:1202 punch from 0x23c4800 to 0x23c7400, (0x2c00 bytes) 2024-09-18T01:03:00.358 INFO:teuthology.orchestra.run.smithi164.stdout:1205 punch from 0xe4aea00 to 0xe4b1c00, (0x3200 bytes) 2024-09-18T01:03:00.364 INFO:teuthology.orchestra.run.smithi164.stdout:1206 trunc from 0xe5de600 to 0xa0cca00 2024-09-18T01:03:00.392 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:00.391+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:00.427 INFO:teuthology.orchestra.run.smithi164.stdout:1207 write 0x8304800 thru 0x8309c84 (0x5485 bytes) 2024-09-18T01:03:00.429 INFO:teuthology.orchestra.run.smithi164.stdout:1209 write 0x7330200 thru 0x7334030 (0x3e31 bytes) 2024-09-18T01:03:00.429 INFO:teuthology.orchestra.run.smithi164.stdout:1211 punch from 0x6fe000 to 0x6ff800, (0x1800 bytes) 2024-09-18T01:03:00.432 INFO:teuthology.orchestra.run.smithi164.stdout:1212 punch from 0x784d800 to 0x7852200, (0x4a00 bytes) 2024-09-18T01:03:00.466 INFO:teuthology.orchestra.run.smithi164.stdout:1215 write 0xbc45400 thru 0xbc50cee (0xb8ef bytes) 2024-09-18T01:03:00.472 INFO:teuthology.orchestra.run.smithi164.stdout:1217 read 0x3b0000 thru 0x3b1ea8 (0x1ea9 bytes) 2024-09-18T01:03:00.472 INFO:teuthology.orchestra.run.smithi164.stdout:1218 read 0x2694e00 thru 0x26a2714 (0xd915 bytes) 2024-09-18T01:03:00.473 INFO:teuthology.orchestra.run.smithi164.stdout:1219 punch from 0x5e91400 to 0x5e97a00, (0x6600 bytes) 2024-09-18T01:03:00.474 INFO:teuthology.orchestra.run.smithi164.stdout:1220 punch from 0x41e4600 to 0x41ef600, (0xb000 bytes) 2024-09-18T01:03:00.475 INFO:teuthology.orchestra.run.smithi164.stdout:1221 punch from 0x47aba00 to 0x47b7000, (0xb600 bytes) 2024-09-18T01:03:00.475 INFO:teuthology.orchestra.run.smithi164.stdout:1222 write 0x9d78800 thru 0x9d869c0 (0xe1c1 bytes) 2024-09-18T01:03:00.475 INFO:teuthology.orchestra.run.smithi164.stdout:1224 read 0x23a7600 thru 0x23b1fec (0xa9ed bytes) 2024-09-18T01:03:00.477 INFO:teuthology.orchestra.run.smithi164.stdout:1225 read 0x6f26800 thru 0x6f282c2 (0x1ac3 bytes) 2024-09-18T01:03:00.477 INFO:teuthology.orchestra.run.smithi164.stdout:1226 read 0x8958600 thru 0x8966125 (0xdb26 bytes) 2024-09-18T01:03:00.477 INFO:teuthology.orchestra.run.smithi164.stdout:1227 write 0x4ec7800 thru 0x4eca218 (0x2a19 bytes) 2024-09-18T01:03:00.478 INFO:teuthology.orchestra.run.smithi164.stdout:1236 write 0x3b92a00 thru 0x3b9df8a (0xb58b bytes) 2024-09-18T01:03:00.478 INFO:teuthology.orchestra.run.smithi164.stdout:1237 punch from 0x10a6c00 to 0x10ab800, (0x4c00 bytes) 2024-09-18T01:03:00.478 INFO:teuthology.orchestra.run.smithi164.stdout:1238 write 0x3a38600 thru 0x3a4784a (0xf24b bytes) 2024-09-18T01:03:00.478 INFO:teuthology.orchestra.run.smithi164.stdout:1239 write 0x83e6c00 thru 0x83e7cb9 (0x10ba bytes) 2024-09-18T01:03:00.479 INFO:teuthology.orchestra.run.smithi164.stdout:1242 write 0x8b98c00 thru 0x8ba028a (0x768b bytes) 2024-09-18T01:03:00.479 INFO:teuthology.orchestra.run.smithi164.stdout:1247 read 0x2d8e800 thru 0x2d9d8f3 (0xf0f4 bytes) 2024-09-18T01:03:00.493 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:00.492+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:00.496 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:00.496 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:00.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:00.536 INFO:teuthology.orchestra.run.smithi164.stdout:1248 trunc from 0xbc50e00 to 0x9f82400 2024-09-18T01:03:00.590 INFO:teuthology.orchestra.run.smithi164.stdout:1251 write 0x13dae00 thru 0x13e1bcd (0x6dce bytes) 2024-09-18T01:03:00.594 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:00.593+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:00.601 INFO:teuthology.orchestra.run.smithi164.stdout:1252 trunc from 0x9f82400 to 0x475a00 2024-09-18T01:03:00.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:00.666 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:00.667 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:00.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:00.694 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:00.693+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:00.769 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:00.770 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:00.781 INFO:teuthology.orchestra.run.smithi164.stdout:1255 punch from 0x226400 to 0x228a00, (0x2600 bytes) 2024-09-18T01:03:00.783 INFO:teuthology.orchestra.run.smithi164.stdout:1256 read 0x283a00 thru 0x288b97 (0x5198 bytes) 2024-09-18T01:03:00.785 INFO:teuthology.orchestra.run.smithi164.stdout:1257 punch from 0x454400 to 0x456800, (0x2400 bytes) 2024-09-18T01:03:00.790 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:00.790 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:00.795 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:00.794+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:00.796 INFO:teuthology.orchestra.run.smithi164.stdout:1258 trunc from 0x475a00 to 0x906b000 2024-09-18T01:03:00.801 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:00.801 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:00.805 INFO:teuthology.orchestra.run.smithi164.stdout:1261 write 0x3b56200 thru 0x3b57f3e (0x1d3f bytes) 2024-09-18T01:03:00.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:00.813 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:00.813 INFO:teuthology.orchestra.run.smithi164.stdout:1262 trunc from 0x906b000 to 0x2e07e00 2024-09-18T01:03:00.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:00.834 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:00.851 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:00.851 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:00.868 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:00.921 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:00.920+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:00.941 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:00.941 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:00.942 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:00.942 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:00.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:01.035 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:01.034+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:01.041 INFO:teuthology.orchestra.run.smithi164.stdout:1264 read 0x11a00 thru 0x1eba3 (0xd1a4 bytes) 2024-09-18T01:03:01.043 INFO:teuthology.orchestra.run.smithi164.stdout:1265 read 0x23a5400 thru 0x23a8c86 (0x3887 bytes) 2024-09-18T01:03:01.043 INFO:teuthology.orchestra.run.smithi164.stdout:1266 read 0x19b000 thru 0x1aa579 (0xf57a bytes) 2024-09-18T01:03:01.044 INFO:teuthology.orchestra.run.smithi164.stdout:1271 punch from 0x1eab600 to 0x1eb5400, (0x9e00 bytes) 2024-09-18T01:03:01.044 INFO:teuthology.orchestra.run.smithi164.stdout:1272 read 0x470800 thru 0x471392 (0xb93 bytes) 2024-09-18T01:03:01.067 INFO:teuthology.orchestra.run.smithi164.stdout:1273 write 0x394b800 thru 0x395b7bb (0xffbc bytes) 2024-09-18T01:03:01.073 INFO:teuthology.orchestra.run.smithi164.stdout:1276 trunc from 0x395b800 to 0x79ff600 2024-09-18T01:03:01.097 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:01.096+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:01.114 INFO:teuthology.orchestra.run.smithi164.stdout:1277 write 0xb9ef000 thru 0xb9ef18a (0x18b bytes) 2024-09-18T01:03:01.120 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:01.120 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2030 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4406272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.317779+0000", 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.8002024909999998, 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01394243, 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2030 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.317779+0000", 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.317779+0000", 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.317780+0000", 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.317783+0000", 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4680000000000001e-06 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.317791+0000", 2024-09-18T01:03:01.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3880000000000002e-06 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.317796+0000", 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7320000000000003e-06 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.329535+0000", 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011739107 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.329606+0000", 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0029000000000003e-05 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.331722+0000", 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021161270000000002 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2045 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4282368~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318963+0000", 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7990189810000001, 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013477741999999999, 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2045 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318963+0000", 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318963+0000", 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318963+0000", 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6899999999999998e-07 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318965+0000", 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2019999999999998e-06 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318973+0000", 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0770000000000008e-06 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318979+0000", 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:01.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332287+0000", 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013308163 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332353+0000", 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6427000000000006e-05 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332440+0000", 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7081000000000002e-05 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2046 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4291584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318986+0000", 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7989956879999998, 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013632684000000001, 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2046 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318986+0000", 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318986+0000", 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318986+0000", 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318989+0000", 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.277e-06 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318997+0000", 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0660000000000004e-06 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319002+0000", 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6230000000000002e-06 2024-09-18T01:03:01.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332468+0000", 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013465711999999999 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332535+0000", 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6441999999999998e-05 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332619+0000", 2024-09-18T01:03:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4166999999999996e-05 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2047 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4300800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.319010+0000", 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7989715779999997, 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013805682, 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2047 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319010+0000", 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319010+0000", 2024-09-18T01:03:01.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319011+0000", 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319013+0000", 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319021+0000", 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5560000000000008e-06 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319027+0000", 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6559999999999997e-06 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332654+0000", 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013627014 2024-09-18T01:03:01.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332722+0000", 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7376e-05 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332816+0000", 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4283000000000005e-05 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2048 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4310016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.319034+0000", 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7989474300000001, 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014151469, 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2048 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319034+0000", 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319034+0000", 2024-09-18T01:03:01.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319035+0000", 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2500000000000001e-07 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319037+0000", 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.097e-06 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319045+0000", 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4330000000000004e-06 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319050+0000", 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.13e-06 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332852+0000", 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013801865999999999 2024-09-18T01:03:01.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.332921+0000", 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8468999999999998e-05 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.333186+0000", 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000265049 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2049 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4319232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.319058+0000", 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.79892385, 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014327830999999999, 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2049 2024-09-18T01:03:01.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319058+0000", 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319058+0000", 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319058+0000", 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319062+0000", 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7400000000000002e-06 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319070+0000", 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0909999999999993e-06 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.319076+0000", 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.502e-06 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.333226+0000", 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014150406000000001 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.333314+0000", 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7614000000000005e-05 2024-09-18T01:03:01.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.333386+0000", 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2082000000000005e-05 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2264 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4200448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322753+0000", 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7952284660000002, 2024-09-18T01:03:01.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013362496, 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2264 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322753+0000", 2024-09-18T01:03:01.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322753+0000", 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0900000000000001e-07 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.252e-06 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322765+0000", 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322786+0000", 2024-09-18T01:03:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1080000000000001e-05 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.335924+0000", 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013137920000000001 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.335995+0000", 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0975000000000004e-05 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336116+0000", 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000120691 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2265 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4209664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322804+0000", 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7951775599999999, 2024-09-18T01:03:01.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013532283000000001, 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2265 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322804+0000", 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322804+0000", 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322805+0000", 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.27e-06 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322816+0000", 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9330000000000001e-06 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322822+0000", 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.767e-06 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336146+0000", 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013324285 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336215+0000", 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8986999999999995e-05 2024-09-18T01:03:01.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336336+0000", 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121441 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2266 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4218880~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7951519359999999, 2024-09-18T01:03:01.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013719873, 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2266 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:01.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322833+0000", 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6290000000000002e-06 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322839+0000", 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322845+0000", 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336366+0000", 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013521386999999999 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336434+0000", 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7571999999999996e-05 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336550+0000", 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116066 2024-09-18T01:03:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2267 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4228096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322860+0000", 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.795121478, 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013899744, 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2267 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322860+0000", 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322860+0000", 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322863+0000", 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.774e-06 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6100000000000006e-06 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322880+0000", 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2290000000000003e-06 2024-09-18T01:03:01.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336578+0000", 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013698119 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336645+0000", 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6903999999999995e-05 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336760+0000", 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114632 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2268 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4237312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322905+0000", 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7950770409999999, 2024-09-18T01:03:01.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014065123000000001, 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2268 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322905+0000", 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322905+0000", 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322905+0000", 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-07 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322915+0000", 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5060000000000001e-06 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322926+0000", 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1063999999999999e-05 2024-09-18T01:03:01.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322932+0000", 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336794+0000", 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013861916 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336864+0000", 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0343000000000001e-05 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336970+0000", 2024-09-18T01:03:01.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105339 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2269 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4246528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322948+0000", 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.795033594, 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014178420000000001, 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2269 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322948+0000", 2024-09-18T01:03:01.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322948+0000", 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322949+0000", 2024-09-18T01:03:01.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-18T01:03:01.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322952+0000", 2024-09-18T01:03:01.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.976e-06 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322960+0000", 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5900000000000008e-06 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322966+0000", 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6679999999999998e-06 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.336999+0000", 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014032902 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337063+0000", 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3695000000000006e-05 2024-09-18T01:03:01.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337127+0000", 2024-09-18T01:03:01.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3962999999999997e-05 2024-09-18T01:03:01.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2270 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4255744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322995+0000", 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7949862539999999, 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014310902, 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2270 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322995+0000", 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322995+0000", 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322996+0000", 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322999+0000", 2024-09-18T01:03:01.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9890000000000001e-06 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323010+0000", 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0766e-05 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323017+0000", 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7300000000000005e-06 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337153+0000", 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014136012 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337224+0000", 2024-09-18T01:03:01.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0247999999999996e-05 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337306+0000", 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.263e-05 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2271 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4264960~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.323032+0000", 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.794949801, 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014446383, 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2271 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323032+0000", 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323032+0000", 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323032+0000", 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323035+0000", 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323042+0000", 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8630000000000002e-06 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323047+0000", 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0359999999999997e-06 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337333+0000", 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01428576 2024-09-18T01:03:01.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337393+0000", 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9908999999999998e-05 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337478+0000", 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5687999999999997e-05 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2272 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4274176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.323055+0000", 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.794926749, 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014634907000000001, 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2272 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323055+0000", 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323055+0000", 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323055+0000", 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323058+0000", 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323064+0000", 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3169999999999998e-06 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323069+0000", 2024-09-18T01:03:01.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8570000000000003e-06 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337509+0000", 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014439689 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337581+0000", 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2107999999999995e-05 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337690+0000", 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010897599999999999 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2273 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4283392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.323077+0000", 2024-09-18T01:03:01.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7949051410000001, 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014800239, 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2273 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323077+0000", 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323077+0000", 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323077+0000", 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3200000000000005e-07 2024-09-18T01:03:01.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323079+0000", 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323085+0000", 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9209999999999997e-06 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323091+0000", 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9880000000000001e-06 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337721+0000", 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014629909 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337793+0000", 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2260000000000003e-05 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337877+0000", 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3380000000000005e-05 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2274 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4292608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.323102+0000", 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7948800350000003, 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014977918999999999, 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2274 2024-09-18T01:03:01.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323102+0000", 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323102+0000", 2024-09-18T01:03:01.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323102+0000", 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1900000000000003e-07 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323105+0000", 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0029999999999999e-06 2024-09-18T01:03:01.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323112+0000", 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9580000000000001e-06 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323117+0000", 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.101e-06 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337906+0000", 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014788279 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.337975+0000", 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9870999999999996e-05 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338080+0000", 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104188 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2275 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4301824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.323125+0000", 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7948571439999998, 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015134467, 2024-09-18T01:03:01.212 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2275 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323125+0000", 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323125+0000", 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323125+0000", 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323127+0000", 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2649999999999999e-06 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323135+0000", 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2640000000000002e-06 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323140+0000", 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338109+0000", 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014968964 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338179+0000", 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9987999999999995e-05 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338259+0000", 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0310999999999997e-05 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2276 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4311040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.323146+0000", 2024-09-18T01:03:01.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7948353940000001, 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015290916, 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2276 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323146+0000", 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323146+0000", 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323147+0000", 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323149+0000", 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0200000000000001e-06 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323155+0000", 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2190000000000001e-06 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323160+0000", 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9969999999999998e-06 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338293+0000", 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015132914000000001 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338359+0000", 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6550999999999994e-05 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338437+0000", 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7831999999999996e-05 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2277 3.8 3:12f8bc67:::rbd_data.110372aa4379.0000000000000004:7 [read 4320256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.323166+0000", 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7948153100000002, 2024-09-18T01:03:01.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015443788, 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2277 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:01.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323166+0000", 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323166+0000", 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323167+0000", 2024-09-18T01:03:01.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9400000000000005e-07 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323169+0000", 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8959999999999999e-06 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323175+0000", 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:03:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.323180+0000", 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338464+0000", 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015284772 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338529+0000", 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4368000000000004e-05 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338610+0000", 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1462e-05 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:01.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:01.230 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:01.231 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:01.230+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:01.233 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:01.233 INFO:teuthology.orchestra.run.smithi164.stdout:1278 write 0xda8c000 thru 0xda95a14 (0x9a15 bytes) 2024-09-18T01:03:01.233 INFO:teuthology.orchestra.run.smithi164.stdout:1279 write 0x189e200 thru 0x189f1e6 (0xfe7 bytes) 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1282 punch from 0x2c60200 to 0x2c65800, (0x5600 bytes) 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1283 read 0x8236e00 thru 0x824673e (0xf93f bytes) 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1284 punch from 0xaae6200 to 0xaaed400, (0x7200 bytes) 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1294 read 0x5751000 thru 0x575e358 (0xd359 bytes) 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1295 trunc from 0xda95c00 to 0x8b71c00 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1296 punch from 0x8273800 to 0x8279a00, (0x6200 bytes) 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1298 trunc from 0x8b71c00 to 0x9b17200 2024-09-18T01:03:01.234 INFO:teuthology.orchestra.run.smithi164.stdout:1303 write 0xada400 thru 0xae5fba (0xbbbb bytes) 2024-09-18T01:03:01.255 INFO:teuthology.orchestra.run.smithi164.stdout:1305 write 0xd4b0600 thru 0xd4b5eea (0x58eb bytes) 2024-09-18T01:03:01.257 INFO:teuthology.orchestra.run.smithi164.stdout:1306 trunc from 0xd4b6000 to 0xc27ba00 2024-09-18T01:03:01.286 INFO:teuthology.orchestra.run.smithi164.stdout:1307 write 0x7104c00 thru 0x7110cd9 (0xc0da bytes) 2024-09-18T01:03:01.288 INFO:teuthology.orchestra.run.smithi164.stdout:1312 write 0x9814800 thru 0x9818ce2 (0x44e3 bytes) 2024-09-18T01:03:01.289 INFO:teuthology.orchestra.run.smithi164.stdout:1315 read 0x332b600 thru 0x3332749 (0x714a bytes) 2024-09-18T01:03:01.290 INFO:teuthology.orchestra.run.smithi164.stdout:1316 trunc from 0xc27ba00 to 0xdbabc00 2024-09-18T01:03:01.321 INFO:teuthology.orchestra.run.smithi164.stdout:1318 write 0x51b7e00 thru 0x51bf275 (0x7476 bytes) 2024-09-18T01:03:01.322 INFO:teuthology.orchestra.run.smithi164.stdout:1319 read 0x3991600 thru 0x3994127 (0x2b28 bytes) 2024-09-18T01:03:01.322 INFO:teuthology.orchestra.run.smithi164.stdout:1321 punch from 0xd0e0a00 to 0xd0e3c00, (0x3200 bytes) 2024-09-18T01:03:01.328 INFO:teuthology.orchestra.run.smithi164.stdout:1322 trunc from 0xdbabc00 to 0x8ff6400 2024-09-18T01:03:01.331 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:01.330+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:01.359 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:01.359 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:01.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19939 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.355611+0000", 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.0018052509999999999, 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0018183889999999999, 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.360 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19939 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355611+0000", 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355611+0000", 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355612+0000", 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7199999999999997e-07 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355616+0000", 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0790000000000002e-06 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355627+0000", 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1138000000000001e-05 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355634+0000", 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0609999999999998e-06 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.363 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4729 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.355633+0000", 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.0017838800000000001, 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001831814, 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.364 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4729 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355633+0000", 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355633+0000", 2024-09-18T01:03:01.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355633+0000", 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1099999999999995e-07 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355636+0000", 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0960000000000001e-06 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355642+0000", 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5729999999999998e-06 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355646+0000", 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6049999999999998e-06 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.367 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4730 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.355922+0000", 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.001494308, 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015708530000000001, 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4730 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.368 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355922+0000", 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355922+0000", 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355923+0000", 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4300000000000003e-07 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355928+0000", 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0479999999999998e-06 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355935+0000", 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1130000000000003e-06 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355939+0000", 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6119999999999999e-06 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19940 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.355917+0000", 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.001499691, 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00160304, 2024-09-18T01:03:01.371 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19940 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355917+0000", 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355917+0000", 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355918+0000", 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8899999999999998e-07 2024-09-18T01:03:01.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355930+0000", 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2074e-05 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355941+0000", 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0812000000000001e-05 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355948+0000", 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.289e-06 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4731 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.356194+0000", 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.001222009, 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013532959999999999, 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.375 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4731 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356194+0000", 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356194+0000", 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356195+0000", 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3e-07 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356198+0000", 2024-09-18T01:03:01.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7199999999999998e-06 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356203+0000", 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8729999999999998e-06 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356206+0000", 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3000000000000002e-06 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19942 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.356210+0000", 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.001206899, 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001366103, 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19942 2024-09-18T01:03:01.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356210+0000", 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356210+0000", 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356211+0000", 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.032e-06 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356216+0000", 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0660000000000003e-06 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356227+0000", 2024-09-18T01:03:01.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1661000000000001e-05 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356234+0000", 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19943 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.356384+0000", 2024-09-18T01:03:01.382 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.0010325390000000001, 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001237284, 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.384 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19943 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356384+0000", 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356384+0000", 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356385+0000", 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5700000000000002e-07 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356389+0000", 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8089999999999999e-06 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356400+0000", 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1022999999999999e-05 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356407+0000", 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0129999999999996e-06 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4732 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.387 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.356399+0000", 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.0010171030000000001, 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012756009999999999, 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4732 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356399+0000", 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356399+0000", 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356400+0000", 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8500000000000001e-07 2024-09-18T01:03:01.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356403+0000", 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5859999999999999e-06 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356407+0000", 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5650000000000003e-06 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356410+0000", 2024-09-18T01:03:01.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2200000000000001e-06 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4733 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.356709+0000", 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.00070726399999999998, 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0010184160000000001, 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.391 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4733 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356709+0000", 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356709+0000", 2024-09-18T01:03:01.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356710+0000", 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.61e-07 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356713+0000", 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.756e-06 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356718+0000", 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1170000000000003e-06 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356721+0000", 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3100000000000001e-06 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.394 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19944 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.356726+0000", 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.00069041299999999996, 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0010559359999999999, 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19944 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356726+0000", 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356726+0000", 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356727+0000", 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.127e-06 2024-09-18T01:03:01.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356732+0000", 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3259999999999997e-06 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356742+0000", 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0829000000000001e-05 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.356749+0000", 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0260000000000002e-06 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19945 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357010+0000", 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.00040686800000000001, 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082426299999999999, 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.398 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19945 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357010+0000", 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357010+0000", 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357010+0000", 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999998e-07 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357015+0000", 2024-09-18T01:03:01.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.07e-06 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357025+0000", 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0813e-05 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357032+0000", 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.737e-06 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.401 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4734 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357021+0000", 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.00039546600000000002, 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00084788799999999998, 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.402 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4734 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357021+0000", 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357021+0000", 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357022+0000", 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0100000000000004e-07 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357025+0000", 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1540000000000002e-06 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357030+0000", 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357034+0000", 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2949999999999998e-06 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4735 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.405 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357422+0000", 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9999943, 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00052578800000000001, 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4735 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357422+0000", 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357422+0000", 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357423+0000", 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7000000000000004e-07 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357426+0000", 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.954e-06 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357430+0000", 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.651e-06 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357434+0000", 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2849999999999999e-06 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.408 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19946 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357418+0000", 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9999986, 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000589764, 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19946 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357418+0000", 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357418+0000", 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357419+0000", 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8700000000000005e-07 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357422+0000", 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7309999999999999e-06 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357431+0000", 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6330000000000002e-06 2024-09-18T01:03:01.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357436+0000", 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.729e-06 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19947 3.f 3.549d26f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357703+0000", 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9997139, 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00036156300000000002, 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19947 2024-09-18T01:03:01.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357703+0000", 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357703+0000", 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357703+0000", 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0100000000000004e-07 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357706+0000", 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9330000000000001e-06 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357713+0000", 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:01.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357718+0000", 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0950000000000003e-06 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19949 3.f 3.8c8c580f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357825+0000", 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9995909, 2024-09-18T01:03:01.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00029345700000000001, 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19949 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357825+0000", 2024-09-18T01:03:01.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357825+0000", 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357826+0000", 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7899999999999998e-07 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357829+0000", 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5919999999999999e-06 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357837+0000", 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5900000000000008e-06 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357841+0000", 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1049999999999997e-06 2024-09-18T01:03:01.419 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19953 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357860+0000", 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9995565, 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00031053100000000001, 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.420 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19953 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357860+0000", 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357860+0000", 2024-09-18T01:03:01.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357860+0000", 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6699999999999999e-07 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357863+0000", 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.413e-06 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357869+0000", 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2330000000000003e-06 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357876+0000", 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5129999999999999e-06 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.423 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4736 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.357895+0000", 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9995217, 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030771599999999998, 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.424 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4736 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357895+0000", 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357895+0000", 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357896+0000", 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.195e-06 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357905+0000", 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1050000000000001e-06 2024-09-18T01:03:01.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357913+0000", 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6040000000000004e-06 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.357926+0000", 2024-09-18T01:03:01.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3427e-05 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:19958 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.358090+0000", 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9993262, 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000141187, 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.428 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 19958 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358090+0000", 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358090+0000", 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358091+0000", 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3399999999999998e-07 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358096+0000", 2024-09-18T01:03:01.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8980000000000004e-06 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358102+0000", 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3350000000000003e-06 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358108+0000", 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.575e-06 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4737 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.358126+0000", 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4294967295.9992905, 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013257699999999999, 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.432 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4737 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358126+0000", 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358126+0000", 2024-09-18T01:03:01.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358127+0000", 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6099999999999999e-07 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358131+0000", 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2780000000000004e-06 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358138+0000", 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9999999999999999e-06 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.358143+0000", 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5299999999999998e-06 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.435 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:4728 3.f 3.549d26f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:01.355496+0000", 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.0019207639999999999, 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027901900000000001, 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 4728 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355496+0000", 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355496+0000", 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355497+0000", 2024-09-18T01:03:01.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7300000000000005e-07 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355501+0000", 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.048e-06 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355509+0000", 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4840000000000004e-06 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:01.355516+0000", 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8079999999999999e-06 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 21 2024-09-18T01:03:01.439 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:01.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:03:01.441 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:01.440+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:01.442 INFO:teuthology.orchestra.run.smithi164.stdout:1324 trunc from 0x8ff6400 to 0x3cab000 2024-09-18T01:03:01.442 INFO:teuthology.orchestra.run.smithi164.stdout:1325 punch from 0x1aba600 to 0x1abb400, (0xe00 bytes) 2024-09-18T01:03:01.442 INFO:teuthology.orchestra.run.smithi164.stdout:1326 flatten 2024-09-18T01:03:01.442 INFO:teuthology.orchestra.run.smithi164.stdout:1328 write 0x62da000 thru 0x62e2cc7 (0x8cc8 bytes) 2024-09-18T01:03:01.442 INFO:teuthology.orchestra.run.smithi164.stdout:1330 read 0x5e15800 thru 0x5e20216 (0xaa17 bytes) 2024-09-18T01:03:01.442 INFO:teuthology.orchestra.run.smithi164.stdout:1331 punch from 0x16f7400 to 0x1700000, (0x8c00 bytes) 2024-09-18T01:03:01.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:01.464 INFO:teuthology.orchestra.run.smithi164.stdout:1332 write 0x6b93e00 thru 0x6ba22e8 (0xe4e9 bytes) 2024-09-18T01:03:01.465 INFO:teuthology.orchestra.run.smithi164.stdout:1335 punch from 0x4549a00 to 0x4550e00, (0x7400 bytes) 2024-09-18T01:03:01.466 INFO:teuthology.orchestra.run.smithi164.stdout:1336 read 0x3de6e00 thru 0x3df4e5a (0xe05b bytes) 2024-09-18T01:03:01.501 INFO:teuthology.orchestra.run.smithi164.stdout:1337 write 0x6cf5800 thru 0x6cff844 (0xa045 bytes) 2024-09-18T01:03:01.502 INFO:teuthology.orchestra.run.smithi164.stdout:1343 read 0x27dd000 thru 0x27de1a5 (0x11a6 bytes) 2024-09-18T01:03:01.509 INFO:teuthology.orchestra.run.smithi164.stdout:1345 trunc from 0x6cffa00 to 0xc413000 2024-09-18T01:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout:1346 write 0xa084600 thru 0xa08728f (0x2c90 bytes) 2024-09-18T01:03:01.535 INFO:teuthology.orchestra.run.smithi164.stdout:1347 write 0x9235600 thru 0x9235cf8 (0x6f9 bytes) 2024-09-18T01:03:01.541 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:01.541+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:01.544 INFO:teuthology.orchestra.run.smithi164.stdout:1349 trunc from 0xc413000 to 0x5446200 2024-09-18T01:03:01.595 INFO:teuthology.orchestra.run.smithi164.stdout:1351 read 0x703000 thru 0x70fcc8 (0xccc9 bytes) 2024-09-18T01:03:01.609 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:01.609 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:01.617 INFO:teuthology.orchestra.run.smithi164.stdout:1353 write 0xe1e1400 thru 0xe1e6712 (0x5313 bytes) 2024-09-18T01:03:01.618 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:01.622 INFO:teuthology.orchestra.run.smithi164.stdout:1355 punch from 0x3fa0400 to 0x3fa8c00, (0x8800 bytes) 2024-09-18T01:03:01.623 INFO:teuthology.orchestra.run.smithi164.stdout:1356 read 0xae46800 thru 0xae49f2f (0x3730 bytes) 2024-09-18T01:03:01.635 INFO:teuthology.orchestra.run.smithi164.stdout:1357 trunc from 0xe1e6800 to 0x4397e00 2024-09-18T01:03:01.643 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:01.642+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:01.672 INFO:teuthology.orchestra.run.smithi164.stdout:1360 read 0x14a7200 thru 0x14a9cd5 (0x2ad6 bytes) 2024-09-18T01:03:01.689 INFO:teuthology.orchestra.run.smithi164.stdout:1362 write 0x861a000 thru 0x8625965 (0xb966 bytes) 2024-09-18T01:03:01.723 INFO:teuthology.orchestra.run.smithi164.stdout:1363 write 0xd45bc00 thru 0xd464d8f (0x9190 bytes) 2024-09-18T01:03:01.729 INFO:teuthology.orchestra.run.smithi164.stdout:1368 write 0x2adcc00 thru 0x2ae84cf (0xb8d0 bytes) 2024-09-18T01:03:01.729 INFO:teuthology.orchestra.run.smithi164.stdout:1370 write 0x97a6600 thru 0x97aef07 (0x8908 bytes) 2024-09-18T01:03:01.729 INFO:teuthology.orchestra.run.smithi164.stdout:1372 punch from 0x2ed5200 to 0x2ed7000, (0x1e00 bytes) 2024-09-18T01:03:01.730 INFO:teuthology.orchestra.run.smithi164.stdout:1373 write 0xa711e00 thru 0xa720156 (0xe357 bytes) 2024-09-18T01:03:01.730 INFO:teuthology.orchestra.run.smithi164.stdout:1375 punch from 0xca0e000 to 0xca11c00, (0x3c00 bytes) 2024-09-18T01:03:01.731 INFO:teuthology.orchestra.run.smithi164.stdout:1376 punch from 0x4112e00 to 0x4120e00, (0xe000 bytes) 2024-09-18T01:03:01.731 INFO:teuthology.orchestra.run.smithi164.stdout:1377 write 0x3e90200 thru 0x3e9f225 (0xf026 bytes) 2024-09-18T01:03:01.731 INFO:teuthology.orchestra.run.smithi164.stdout:1378 read 0xb4dd000 thru 0xb4ebfed (0xefee bytes) 2024-09-18T01:03:01.732 INFO:teuthology.orchestra.run.smithi164.stdout:1379 punch from 0x5d52800 to 0x5d62600, (0xfe00 bytes) 2024-09-18T01:03:01.732 INFO:teuthology.orchestra.run.smithi164.stdout:1381 read 0x8431800 thru 0x84407e0 (0xefe1 bytes) 2024-09-18T01:03:01.732 INFO:teuthology.orchestra.run.smithi164.stdout:1383 write 0x47ba600 thru 0x47c5e25 (0xb826 bytes) 2024-09-18T01:03:01.732 INFO:teuthology.orchestra.run.smithi164.stdout:1384 write 0xf16a00 thru 0xf18567 (0x1b68 bytes) 2024-09-18T01:03:01.740 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1975 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638548+0000", 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0999973369999996, 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016067886, 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1975 2024-09-18T01:03:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638548+0000", 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638548+0000", 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638549+0000", 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638550+0000", 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638556+0000", 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3750000000000002e-06 2024-09-18T01:03:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638560+0000", 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4139999999999996e-06 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654612+0000", 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016051488999999999 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654616+0000", 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4410000000000003e-06 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1976 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638566+0000", 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.099979716, 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016073253999999999, 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1976 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638566+0000", 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638566+0000", 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638566+0000", 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8000000000000001e-07 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638568+0000", 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638573+0000", 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1819999999999997e-06 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638578+0000", 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.746e-06 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654635+0000", 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016056714 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654639+0000", 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1309999999999999e-06 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1978 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638583+0000", 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0999620950000004, 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016076100999999999, 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1978 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638583+0000", 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638583+0000", 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638584+0000", 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2e-07 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638586+0000", 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.824e-06 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638591+0000", 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3129999999999998e-06 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638595+0000", 2024-09-18T01:03:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2309999999999998e-06 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654655+0000", 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016060075 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654660+0000", 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2379999999999999e-06 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1979 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638601+0000", 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.099944678, 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016082727000000002, 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1979 2024-09-18T01:03:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638601+0000", 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638601+0000", 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638601+0000", 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7e-07 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638604+0000", 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5770000000000001e-06 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638609+0000", 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6960000000000002e-06 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638613+0000", 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9700000000000001e-06 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654676+0000", 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016062944999999999 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654684+0000", 2024-09-18T01:03:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1690000000000003e-06 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1980 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638619+0000", 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0999265660000006, 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016083084000000001, 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1980 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638619+0000", 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638619+0000", 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638619+0000", 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9799999999999999e-07 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638621+0000", 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0439999999999998e-06 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638627+0000", 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8019999999999997e-06 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638632+0000", 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5979999999999999e-06 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654696+0000", 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016064522000000001 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654702+0000", 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7200000000000003e-06 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1984 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638692+0000", 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0998531329999999, 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016078035000000001, 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1984 2024-09-18T01:03:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638692+0000", 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638692+0000", 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638693+0000", 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.77e-07 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638695+0000", 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.939e-06 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638700+0000", 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6130000000000003e-06 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638704+0000", 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1489999999999996e-06 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654761+0000", 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016056886999999999 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654770+0000", 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.0699999999999996e-06 2024-09-18T01:03:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1985 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638710+0000", 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0998358649999993, 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016083061999999999, 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1985 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638710+0000", 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638710+0000", 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638710+0000", 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5199999999999998e-07 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638712+0000", 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0080000000000001e-06 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638717+0000", 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3700000000000003e-06 2024-09-18T01:03:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638722+0000", 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4710000000000001e-06 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654789+0000", 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016066668999999999 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654793+0000", 2024-09-18T01:03:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1986 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638727+0000", 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0998182179999993, 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016093824999999999, 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1986 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638727+0000", 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638727+0000", 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638728+0000", 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3599999999999999e-07 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638730+0000", 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9139999999999998e-06 2024-09-18T01:03:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638736+0000", 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6359999999999997e-06 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638741+0000", 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.259e-06 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654816+0000", 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016074578999999999 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654821+0000", 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.101e-06 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1987 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638748+0000", 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0997979050000009, 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016097909000000001, 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1987 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638748+0000", 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638748+0000", 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638748+0000", 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638755+0000", 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8499999999999996e-06 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638761+0000", 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8839999999999999e-06 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638767+0000", 2024-09-18T01:03:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2859999999999996e-06 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654842+0000", 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016074534000000001 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654846+0000", 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9580000000000001e-06 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1988 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638774+0000", 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0997717940000005, 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016094944999999999, 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1988 2024-09-18T01:03:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638774+0000", 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638774+0000", 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638774+0000", 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638776+0000", 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1660000000000001e-06 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638782+0000", 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2650000000000004e-06 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638786+0000", 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5950000000000001e-06 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654865+0000", 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016078623 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654869+0000", 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8369999999999999e-06 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1989 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638817+0000", 2024-09-18T01:03:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0997290230000001, 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016078413999999999, 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1989 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638817+0000", 2024-09-18T01:03:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638817+0000", 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638817+0000", 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3399999999999999e-07 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638819+0000", 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0640000000000001e-06 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638825+0000", 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8880000000000002e-06 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638829+0000", 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4789999999999998e-06 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654887+0000", 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016057159000000001 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654895+0000", 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2900000000000002e-06 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1990 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638836+0000", 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0997097880000002, 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016103749000000001, 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1990 2024-09-18T01:03:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638836+0000", 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638836+0000", 2024-09-18T01:03:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638836+0000", 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3799999999999998e-07 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638838+0000", 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0439999999999998e-06 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638844+0000", 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.643e-06 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638848+0000", 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3769999999999998e-06 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654934+0000", 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016085606999999998 2024-09-18T01:03:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654939+0000", 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6400000000000002e-06 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1991 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638854+0000", 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0996915170000001, 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016117797, 2024-09-18T01:03:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1991 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638854+0000", 2024-09-18T01:03:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638854+0000", 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638854+0000", 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9900000000000001e-07 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638865+0000", 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0633e-05 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638872+0000", 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.007e-06 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638876+0000", 2024-09-18T01:03:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4290000000000003e-06 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654967+0000", 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016090808000000002 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654972+0000", 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5210000000000004e-06 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1992 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638884+0000", 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0996610459999996, 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016114092, 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1992 2024-09-18T01:03:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638884+0000", 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638884+0000", 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638885+0000", 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.96e-07 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638887+0000", 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638893+0000", 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6749999999999999e-06 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638897+0000", 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5900000000000001e-06 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654993+0000", 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016095293 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.654999+0000", 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9429999999999997e-06 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1993 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638903+0000", 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0996425779999992, 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016119112000000001, 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1993 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638903+0000", 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638903+0000", 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638903+0000", 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8099999999999998e-07 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638905+0000", 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.928e-06 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638911+0000", 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3929999999999999e-06 2024-09-18T01:03:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638915+0000", 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3959999999999999e-06 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655018+0000", 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016103035000000002 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655022+0000", 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9790000000000004e-06 2024-09-18T01:03:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1995 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638925+0000", 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0996209910000001, 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016127794000000001, 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1995 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638925+0000", 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638925+0000", 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638925+0000", 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638927+0000", 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638933+0000", 2024-09-18T01:03:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1419999999999998e-06 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638938+0000", 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0170000000000004e-06 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655047+0000", 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016108331 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655052+0000", 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.789e-06 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1996 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638944+0000", 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0996014009999993, 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016130413999999999, 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1996 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638944+0000", 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638944+0000", 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638945+0000", 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0400000000000002e-07 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638947+0000", 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638953+0000", 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0990000000000004e-06 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638958+0000", 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9459999999999997e-06 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655071+0000", 2024-09-18T01:03:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016112906999999999 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655075+0000", 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9580000000000001e-06 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1997 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.638963+0000", 2024-09-18T01:03:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0995820720000005, 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016134483000000002, 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1997 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638963+0000", 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638963+0000", 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638964+0000", 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5499999999999999e-07 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638966+0000", 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0049999999999999e-06 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638972+0000", 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7350000000000001e-06 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.638983+0000", 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1119e-05 2024-09-18T01:03:01.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655093+0000", 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016110512 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655098+0000", 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7570000000000004e-06 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1998 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.639006+0000", 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0995395299999995, 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016113299000000001, 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1998 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.820 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639006+0000", 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639006+0000", 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639006+0000", 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5900000000000002e-07 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639009+0000", 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2759999999999999e-06 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639015+0000", 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1650000000000003e-06 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639020+0000", 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8319999999999996e-06 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655115+0000", 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016095577 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655119+0000", 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9899999999999999e-06 2024-09-18T01:03:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4608.0:1999 3.15 3.f173ba35 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:53.639026+0000", 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.0995196650000008, 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016113565, 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4608", 2024-09-18T01:03:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/192281627", 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1999 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639026+0000", 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639026+0000", 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639026+0000", 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3799999999999997e-07 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639028+0000", 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9410000000000001e-06 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639034+0000", 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5199999999999997e-06 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.639038+0000", 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.464e-06 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655135+0000", 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.016097172 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:53.655139+0000", 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999996e-06 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:01.828 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:01.829 INFO:teuthology.orchestra.run.smithi164.stdout:1385 trunc from 0xd464e00 to 0x4a88600 2024-09-18T01:03:01.830 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:01.830 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:01.828+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:01.875 INFO:teuthology.orchestra.run.smithi164.stdout:1386 write 0x1b11600 thru 0x1b1daa3 (0xc4a4 bytes) 2024-09-18T01:03:01.903 INFO:teuthology.orchestra.run.smithi164.stdout:1387 write 0x7d37c00 thru 0x7d3f3b2 (0x77b3 bytes) 2024-09-18T01:03:01.905 INFO:teuthology.orchestra.run.smithi164.stdout:1390 write 0x6df3600 thru 0x6df56ec (0x20ed bytes) 2024-09-18T01:03:01.906 INFO:teuthology.orchestra.run.smithi164.stdout:1392 write 0x63d7c00 thru 0x63de781 (0x6b82 bytes) 2024-09-18T01:03:01.906 INFO:teuthology.orchestra.run.smithi164.stdout:1394 write 0x2b38e00 thru 0x2b3df96 (0x5197 bytes) 2024-09-18T01:03:01.906 INFO:teuthology.orchestra.run.smithi164.stdout:1395 read 0x25e6200 thru 0x25e95ed (0x33ee bytes) 2024-09-18T01:03:01.906 INFO:teuthology.orchestra.run.smithi164.stdout:1396 punch from 0x5229a00 to 0x522a000, (0x600 bytes) 2024-09-18T01:03:01.915 INFO:teuthology.orchestra.run.smithi164.stdout:1397 trunc from 0x7d3f400 to 0x253a00 2024-09-18T01:03:01.930 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:01.929+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:02.010 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:02.011 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:02.023 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:02.031 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:02.030+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:02.132 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:02.131+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:02.140 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:02.140 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:02.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:02.181 INFO:teuthology.orchestra.run.smithi164.stdout:1398 read 0x169200 thru 0x177703 (0xe504 bytes) 2024-09-18T01:03:02.194 INFO:teuthology.orchestra.run.smithi164.stdout:1399 trunc from 0x253a00 to 0x8d80e00 2024-09-18T01:03:02.205 INFO:teuthology.orchestra.run.smithi164.stdout:1400 punch from 0x14f4a00 to 0x1503400, (0xea00 bytes) 2024-09-18T01:03:02.232 INFO:teuthology.orchestra.run.smithi164.stdout:1405 write 0xbdc3c00 thru 0xbdcae93 (0x7294 bytes) 2024-09-18T01:03:02.232 INFO:teuthology.orchestra.run.smithi164.stdout:1406 read 0x4039600 thru 0x4040b63 (0x7564 bytes) 2024-09-18T01:03:02.233 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:02.232+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:02.244 INFO:teuthology.orchestra.run.smithi164.stdout:1409 trunc from 0xbdcb000 to 0x2124800 2024-09-18T01:03:02.265 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2102 3.1c 3:3b1acae3:::rbd_data.110372aa4379.0000000000000006:7 [read 4155392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.317717+0000", 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9452909840000001, 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017328367000000001, 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2102 2024-09-18T01:03:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317717+0000", 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317717+0000", 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317717+0000", 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0800000000000001e-07 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317718+0000", 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9099999999999991e-07 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317721+0000", 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.067e-06 2024-09-18T01:03:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317723+0000", 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.334889+0000", 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017165782000000001 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.334963+0000", 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3932999999999998e-05 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335045+0000", 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2093000000000002e-05 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2103 3.1c 3:3b1acae3:::rbd_data.110372aa4379.0000000000000006:7 [read 4164608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.317729+0000", 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9452790439999998, 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017488593, 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2103 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317729+0000", 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317729+0000", 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317729+0000", 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5699999999999999e-07 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317730+0000", 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.235e-06 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317734+0000", 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2849999999999999e-06 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317736+0000", 2024-09-18T01:03:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3810000000000002e-06 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335073+0000", 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017336707 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335137+0000", 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3928000000000002e-05 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335217+0000", 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0799999999999999e-05 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2104 3.1c 3:3b1acae3:::rbd_data.110372aa4379.0000000000000006:7 [read 4173824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.317739+0000", 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9452686830000001, 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017683563999999999, 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2104 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317739+0000", 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317739+0000", 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317739+0000", 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3999999999999998e-07 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317740+0000", 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0240000000000001e-06 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317745+0000", 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.25e-06 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317749+0000", 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0010000000000003e-06 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335255+0000", 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017505847000000001 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335332+0000", 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7781000000000006e-05 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335423+0000", 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.0420999999999994e-05 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2105 3.1c 3:3b1acae3:::rbd_data.110372aa4379.0000000000000006:7 [read 4183040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.317768+0000", 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.945240267, 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017823439, 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2105 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317768+0000", 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317768+0000", 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317768+0000", 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1200000000000003e-07 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317770+0000", 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9300000000000002e-06 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317776+0000", 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7880000000000003e-06 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317782+0000", 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7100000000000004e-06 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335450+0000", 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017668902 2024-09-18T01:03:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335513+0000", 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2173000000000003e-05 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335591+0000", 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8424000000000007e-05 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2106 3.1c 3:3b1acae3:::rbd_data.110372aa4379.0000000000000006:7 [read 4192256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.317925+0000", 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9450832299999998, 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017890573999999999, 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2106 2024-09-18T01:03:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317925+0000", 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317925+0000", 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317926+0000", 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0550000000000001e-06 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317931+0000", 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9899999999999997e-06 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317945+0000", 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3995999999999999e-05 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.317952+0000", 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335629+0000", 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017677385 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335711+0000", 2024-09-18T01:03:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1877000000000002e-05 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.335815+0000", 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010393 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2128 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4116480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.318987+0000", 2024-09-18T01:03:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9440205490000002, 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017453968, 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2128 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.318987+0000", 2024-09-18T01:03:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.318987+0000", 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.318988+0000", 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9299999999999998e-07 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.318990+0000", 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2060000000000001e-06 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.318999+0000", 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5110000000000003e-06 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319003+0000", 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8829999999999997e-06 2024-09-18T01:03:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336285+0000", 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017281225000000001 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336351+0000", 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6088999999999997e-05 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336441+0000", 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.0561000000000002e-05 2024-09-18T01:03:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2129 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4134912~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.319013+0000", 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9439944689999997, 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017636122000000001, 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2129 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319013+0000", 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319013+0000", 2024-09-18T01:03:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319014+0000", 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319016+0000", 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1950000000000002e-06 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319023+0000", 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2540000000000003e-06 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319028+0000", 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8690000000000003e-06 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336481+0000", 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017453303 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336553+0000", 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1680000000000005e-05 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336649+0000", 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.6409999999999993e-05 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2130 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4144128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.319035+0000", 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.943972772, 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017823874999999999, 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2130 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319035+0000", 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319035+0000", 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319036+0000", 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4000000000000002e-07 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319038+0000", 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0820000000000001e-06 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319044+0000", 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5180000000000001e-06 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319049+0000", 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6609999999999999e-06 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336694+0000", 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017645151000000001 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336770+0000", 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5801000000000002e-05 2024-09-18T01:03:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336859+0000", 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9221999999999994e-05 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2131 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4153344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.319055+0000", 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9439531519999997, 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017972460999999999, 2024-09-18T01:03:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2131 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319055+0000", 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319055+0000", 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319055+0000", 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2199999999999999e-07 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319063+0000", 2024-09-18T01:03:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9980000000000003e-06 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319070+0000", 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.844e-06 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319075+0000", 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7280000000000003e-06 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336902+0000", 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017826814 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.336966+0000", 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4246999999999997e-05 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337027+0000", 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1408000000000006e-05 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2132 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4162560~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.319082+0000", 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.943926034, 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018147641999999999, 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2132 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319082+0000", 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319082+0000", 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319082+0000", 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0100000000000005e-07 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319084+0000", 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1449999999999998e-06 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319091+0000", 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2469999999999997e-06 2024-09-18T01:03:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319095+0000", 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4519999999999999e-06 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337062+0000", 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017966975 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337135+0000", 2024-09-18T01:03:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2863999999999999e-05 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337229+0000", 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4457999999999996e-05 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2133 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4171776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.319105+0000", 2024-09-18T01:03:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9439024539999998, 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018357945000000001, 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2133 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319105+0000", 2024-09-18T01:03:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319105+0000", 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319106+0000", 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-07 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319108+0000", 2024-09-18T01:03:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3580000000000001e-06 2024-09-18T01:03:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319115+0000", 2024-09-18T01:03:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9619999999999996e-06 2024-09-18T01:03:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319120+0000", 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9590000000000003e-06 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337264+0000", 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018144219999999999 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337351+0000", 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6139999999999999e-05 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337463+0000", 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011278200000000001 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2134 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4180992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.319127+0000", 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.943880901, 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018538240000000001, 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2134 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319127+0000", 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319127+0000", 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319127+0000", 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319130+0000", 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1770000000000001e-06 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319136+0000", 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4609999999999996e-06 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319141+0000", 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6530000000000002e-06 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337502+0000", 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018361077 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337589+0000", 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6975e-05 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337665+0000", 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6416999999999997e-05 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:2135 3.e 3:76859732:::rbd_data.110372aa4379.0000000000000007:7 [read 4190208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.319147+0000", 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.9438608240000002, 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018710385999999999, 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2135 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319147+0000", 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319147+0000", 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319147+0000", 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3500000000000002e-07 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319149+0000", 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9709999999999998e-06 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319156+0000", 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2310000000000001e-06 2024-09-18T01:03:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.319167+0000", 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1187e-05 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337702+0000", 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018535454999999999 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337772+0000", 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9635999999999995e-05 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.337857+0000", 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5570999999999998e-05 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:3674 3.1d 3:bce589a2:::rbd_data.1103a6f5bae0.0000000000000001:8 [read 70656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.529679+0000", 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.7333287500000001, 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017406834999999999, 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3674 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529679+0000", 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529679+0000", 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529680+0000", 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5999999999999999e-07 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529682+0000", 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.024e-06 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529688+0000", 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9660000000000001e-06 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529692+0000", 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9359999999999998e-06 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.546889+0000", 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017196131 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.546980+0000", 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547086+0000", 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000105702 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:3675 3.1d 3:bce589a2:::rbd_data.1103a6f5bae0.0000000000000001:8 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.529699+0000", 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.7333087850000002, 2024-09-18T01:03:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017516758, 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3675 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529699+0000", 2024-09-18T01:03:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529699+0000", 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529699+0000", 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5400000000000002e-07 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529701+0000", 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529707+0000", 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0109999999999997e-06 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529713+0000", 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1850000000000003e-06 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547114+0000", 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017401100999999999 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547162+0000", 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8446999999999999e-05 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547216+0000", 2024-09-18T01:03:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3640999999999997e-05 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:3676 3.1d 3:bce589a2:::rbd_data.1103a6f5bae0.0000000000000001:8 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.529718+0000", 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.7332902570000002, 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017737018, 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3676 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529718+0000", 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529718+0000", 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529718+0000", 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7000000000000001e-07 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529719+0000", 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3260000000000001e-06 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529723+0000", 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.585e-06 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529725+0000", 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6000000000000001e-06 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547259+0000", 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017533554999999999 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547348+0000", 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9097000000000004e-05 2024-09-18T01:03:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547455+0000", 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000106585 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:3677 3.1d 3:bce589a2:::rbd_data.1103a6f5bae0.0000000000000001:8 [read 98304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.529728+0000", 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.7332795839999999, 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017857680000000001, 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3677 2024-09-18T01:03:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529728+0000", 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529728+0000", 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529728+0000", 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2100000000000001e-07 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529730+0000", 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1090000000000001e-06 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529733+0000", 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1e-06 2024-09-18T01:03:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529735+0000", 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5100000000000001e-06 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547482+0000", 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017746921999999998 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547532+0000", 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0318999999999997e-05 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547586+0000", 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3498999999999997e-05 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:3678 3.1d 3:bce589a2:::rbd_data.1103a6f5bae0.0000000000000001:8 [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.529738+0000", 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.733269704, 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018089905, 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3678 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529738+0000", 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529738+0000", 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529738+0000", 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529739+0000", 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.065e-06 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529743+0000", 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0979999999999998e-06 2024-09-18T01:03:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529745+0000", 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.503e-06 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547627+0000", 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.017881636999999999 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547714+0000", 2024-09-18T01:03:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7658000000000007e-05 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547828+0000", 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000113727 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:3679 3.1d 3:bce589a2:::rbd_data.1103a6f5bae0.0000000000000001:8 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.529748+0000", 2024-09-18T01:03:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.733259968, 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018215833000000001, 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3679 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529748+0000", 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529748+0000", 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529748+0000", 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.03e-07 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529749+0000", 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0270000000000001e-06 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529752+0000", 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0230000000000001e-06 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529755+0000", 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.599e-06 2024-09-18T01:03:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547854+0000", 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018099609999999999 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547901+0000", 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6317999999999999e-05 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547964+0000", 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3052999999999994e-05 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:3680 3.1d 3:bce589a2:::rbd_data.1103a6f5bae0.0000000000000001:8 [read 125952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:58.529758+0000", 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 3.733250204, 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018408724000000001, 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 3680 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529758+0000", 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529758+0000", 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529758+0000", 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529759+0000", 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.052e-06 2024-09-18T01:03:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529762+0000", 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1109999999999999e-06 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.529765+0000", 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2329999999999998e-06 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.547997+0000", 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.018232148 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.548085+0000", 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7120000000000006e-05 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:58.548166+0000", 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1849999999999997e-05 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.362 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:02.363 INFO:teuthology.orchestra.run.smithi164.stdout:1411 write 0xc3b1000 thru 0xc3c0699 (0xf69a bytes) 2024-09-18T01:03:02.363 INFO:teuthology.orchestra.run.smithi164.stdout:1413 trunc from 0xc3c0800 to 0x1a5a400 2024-09-18T01:03:02.364 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:03:02.364 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:02.365 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:02.362+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:02.464 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:02.463+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:02.467 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:02.467 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:02.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:02.565 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:02.564+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:02.578 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:02.578 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:02.578 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:02.578 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:02.578 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:02.587 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:02.611 INFO:teuthology.orchestra.run.smithi164.stdout:1414 read 0x2dc600 thru 0x2dd227 (0xc28 bytes) 2024-09-18T01:03:02.612 INFO:teuthology.orchestra.run.smithi164.stdout:1419 punch from 0xba2400 to 0xba6600, (0x4200 bytes) 2024-09-18T01:03:02.613 INFO:teuthology.orchestra.run.smithi164.stdout:1420 punch from 0x48600 to 0x4ee00, (0x6800 bytes) 2024-09-18T01:03:02.613 INFO:teuthology.orchestra.run.smithi164.stdout:1422 read 0xac1600 thru 0xacafc8 (0x99c9 bytes) 2024-09-18T01:03:02.613 INFO:teuthology.orchestra.run.smithi164.stdout:1425 read 0x964600 thru 0x9689e0 (0x43e1 bytes) 2024-09-18T01:03:02.613 INFO:teuthology.orchestra.run.smithi164.stdout:1429 read 0x61f400 thru 0x61f702 (0x303 bytes) 2024-09-18T01:03:02.613 INFO:teuthology.orchestra.run.smithi164.stdout:1433 read 0x4e3800 thru 0x4ec4c2 (0x8cc3 bytes) 2024-09-18T01:03:02.636 INFO:teuthology.orchestra.run.smithi164.stdout:1440 write 0x213d000 thru 0x213e575 (0x1576 bytes) 2024-09-18T01:03:02.636 INFO:teuthology.orchestra.run.smithi164.stdout:1441 punch from 0x707a00 to 0x70cc00, (0x5200 bytes) 2024-09-18T01:03:02.638 INFO:teuthology.orchestra.run.smithi164.stdout:1444 read 0x738600 thru 0x73f061 (0x6a62 bytes) 2024-09-18T01:03:02.641 INFO:teuthology.orchestra.run.smithi164.stdout:1445 trunc from 0x213e600 to 0x4cfe800 2024-09-18T01:03:02.671 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:02.670+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:02.684 INFO:teuthology.orchestra.run.smithi164.stdout:1446 write 0xc116e00 thru 0xc11704a (0x24b bytes) 2024-09-18T01:03:02.708 INFO:teuthology.orchestra.run.smithi164.stdout:1449 write 0xe2d8e00 thru 0xe2e27b1 (0x99b2 bytes) 2024-09-18T01:03:02.709 INFO:teuthology.orchestra.run.smithi164.stdout:1450 read 0xcd7da00 thru 0xcd86acb (0x90cc bytes) 2024-09-18T01:03:02.709 INFO:teuthology.orchestra.run.smithi164.stdout:1453 write 0xdba5000 thru 0xdbab08f (0x6090 bytes) 2024-09-18T01:03:02.709 INFO:teuthology.orchestra.run.smithi164.stdout:1454 read 0xa5ffe00 thru 0xa6033d8 (0x35d9 bytes) 2024-09-18T01:03:02.709 INFO:teuthology.orchestra.run.smithi164.stdout:1455 punch from 0x456c400 to 0x456da00, (0x1600 bytes) 2024-09-18T01:03:02.713 INFO:teuthology.orchestra.run.smithi164.stdout:1456 read 0xd48bc00 thru 0xd490d36 (0x5137 bytes) 2024-09-18T01:03:02.717 INFO:teuthology.orchestra.run.smithi164.stdout:1457 trunc from 0xe2e2800 to 0xabed800 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 30.368358681, 2024-09-18T01:03:02.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:02.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:02.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 29.370347376000002, 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.752 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:02.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:02.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:02.758 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 28.362670424000001, 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.759 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:02.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:02.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.765 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 19.264812473999999, 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.766 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:02.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.772 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.408731462, 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.773 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:02.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:02.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.408362166, 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.779 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:02.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:02.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.406623321, 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.785 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:02.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.789 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.406551431, 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:02.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:02.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:02.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.318685421, 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.794 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:02.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:02.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:02.800 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.312494162, 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:02.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:02.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.310964477000001, 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.805 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:02.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.310584788, 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:02.811 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:02.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:02.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.815 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.295086384999999, 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.818 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.821 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:02.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:02.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.294879626, 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:02.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:02.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.832 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.294461868999999, 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:02.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.834 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:02.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.294252630999999, 2024-09-18T01:03:02.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:02.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.841 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.294188696999999, 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:02.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:02.846 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.293440492, 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:02.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.034501687000001, 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.851 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:02.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 15.033638069, 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:02.857 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:02.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:02.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:02.863 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:02.866 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:02.863+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:02.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:02.866 INFO:teuthology.orchestra.run.smithi164.stdout:1458 read 0xa630600 thru 0xa6326c4 (0x20c5 bytes) 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1461 punch from 0x80dcc00 to 0x80dce00, (0x200 bytes) 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1462 read 0x5d32e00 thru 0x5d34869 (0x1a6a bytes) 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1463 write 0x30ac00 thru 0x3109d6 (0x5dd7 bytes) 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1464 trunc from 0xabed800 to 0x8193000 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1465 read 0x7fdca00 thru 0x7fdcb80 (0x181 bytes) 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1467 trunc from 0x8193000 to 0xbe44400 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1468 write 0x60ea200 thru 0x60f17b1 (0x75b2 bytes) 2024-09-18T01:03:02.867 INFO:teuthology.orchestra.run.smithi164.stdout:1469 clone 7 order 18 su 16384 sc 9 2024-09-18T01:03:02.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:02.965 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:02.964+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:02.975 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:02.975 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:02.975 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:02.975 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:02.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:03.034 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:03.034 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:03.052 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:03.052 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:03.066 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:03.065+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:03.068 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:03.069 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:03.085 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:03.086 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:03.092 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:03.093 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:03.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:03.102 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:03.103 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:03.119 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:03.119 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:03.173 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:03.172+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:03.210 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:03.210 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:03.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:03.268 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:03.267+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:03.369 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:03.367+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:03.372 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:03.372 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:03.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:03.470 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:03.469+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:03.486 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:03.487 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:03.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:03.571 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:03.570+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:03.644 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:03.644 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:03.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:03.697 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:03.696+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:03.769 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:03.769 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:03.772 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:03.771+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:03.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:03.857 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-18T01:03:03.873 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:03.872+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:03.927 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:03.927 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:03.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:03.953 INFO:teuthology.orchestra.run.smithi164.stdout:1470 clone 8 order 22 su 65536 sc 6 2024-09-18T01:03:03.974 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:03.972+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:04.075 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:04.074+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:04.083 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:04.083 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:04.091 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:04.176 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:04.174+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:04.199 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:04.199 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:04.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:04.277 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:04.276+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:04.351 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:04.352 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:04.359 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:04.359 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:04.360 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T01:03:04.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T01:03:04.364 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:04.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:03:04.409 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:04.408+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:04.429 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone7 from 0xbe44400 (overlap 0xbe44400) to 0x9836800 2024-09-18T01:03:04.478 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:04.477+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:04.499 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:04.499 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:04.509 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:04.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T01:03:04.635 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:04.633+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:04.722 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:04.721+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:04.741 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:04.741 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:04.750 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:04.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T01:03:04.821 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:04.820+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:04.857 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-18T01:03:04.860 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:04.860 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:04.868 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:04.922 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:04.921+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:04.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T01:03:05.022 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:05.021+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:05.045 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:05.045 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:05.055 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:05.123 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:05.122+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:05.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:05.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T01:03:05.216 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:05.217 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:05.224 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:05.223+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:05.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:05.312 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:05.312 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:05.324 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:05.323+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:05.334 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:05.361 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:05.361 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:03:05.396 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:05.397 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:05.399 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:05.399 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:05.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:05.426 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:05.450 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:05.449+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:05.452 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:05.526 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:05.525+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:05.559 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:05.559 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:05.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:05.559 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.183226638000001, 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:05.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:05.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:05.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.566 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.185215333000002, 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:05.567 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:05.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:05.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:05.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:05.574 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.177538381000002, 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:05.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:05.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:05.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:05.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:05.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.079680431, 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.582 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:05.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:05.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:05.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.223599418999999, 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.589 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:05.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:05.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:05.595 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.223230123, 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:05.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:05.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:05.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:05.601 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.221491277999998, 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.221419388000001, 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.606 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.610 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.133553377999998, 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:05.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:05.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.127362119000001, 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:05.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.125832433999999, 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.622 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:05.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:05.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.628 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.125452745, 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:05.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:05.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.634 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.109954342000002, 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:05.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:05.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:05.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.109747583000001, 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:05.641 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:05.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:05.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.109329826, 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.647 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:05.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:05.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.109120588, 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:05.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:05.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.109056654, 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.656 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:05.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 18.108308448999999, 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.661 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:05.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:05.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.665 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.849369643999999, 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.666 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:05.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:05.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:05.672 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.848506025999999, 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:05.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:05.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:05.675 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:05.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:05.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:05.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:05.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:05.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:05.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:05.681 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:05.682 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:05.681+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:05.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:05.783 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:05.782+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:05.804 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:05.804 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:05.804 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:05.804 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:05.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:05.884 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:05.883+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:05.916 INFO:teuthology.orchestra.run.smithi164.stdout:1473 write 0x4fd6400 thru 0x4fe061f (0xa220 bytes) 2024-09-18T01:03:05.917 INFO:teuthology.orchestra.run.smithi164.stdout:1475 write 0x86f7200 thru 0x86f7e72 (0xc73 bytes) 2024-09-18T01:03:05.918 INFO:teuthology.orchestra.run.smithi164.stdout:1476 write 0x9800c00 thru 0x9804889 (0x3c8a bytes) 2024-09-18T01:03:05.919 INFO:teuthology.orchestra.run.smithi164.stdout:1477 read 0x4e3ea00 thru 0x4e45cad (0x72ae bytes) 2024-09-18T01:03:05.921 INFO:teuthology.orchestra.run.smithi164.stdout:1480 read 0xba40e00 thru 0xba49793 (0x8994 bytes) 2024-09-18T01:03:05.921 INFO:teuthology.orchestra.run.smithi164.stdout:1481 write 0xa27b000 thru 0xa283193 (0x8194 bytes) 2024-09-18T01:03:05.922 INFO:teuthology.orchestra.run.smithi164.stdout:1485 punch from 0x496e800 to 0x497d800, (0xf000 bytes) 2024-09-18T01:03:05.962 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:05.962 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:05.967 INFO:teuthology.orchestra.run.smithi164.stdout:1487 read 0x34b9800 thru 0x34bdb64 (0x4365 bytes) 2024-09-18T01:03:05.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:05.977 INFO:teuthology.orchestra.run.smithi164.stdout:1488 write 0xc878c00 thru 0xc886caa (0xe0ab bytes) 2024-09-18T01:03:06.000 INFO:teuthology.orchestra.run.smithi164.stdout:1490 write 0xe62ec00 thru 0xe63519d (0x659e bytes) 2024-09-18T01:03:06.010 INFO:teuthology.orchestra.run.smithi164.stdout:1491 trunc from 0xe635200 to 0x752f600 2024-09-18T01:03:06.015 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:06.014+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:06.086 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:06.085+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:06.099 INFO:teuthology.orchestra.run.smithi164.stdout:1496 read 0x55a5a00 thru 0x55b3d85 (0xe386 bytes) 2024-09-18T01:03:06.106 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:06.106 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:06.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:06.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2015 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4265984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7859330690000004, 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027955569999999999, 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2015 2024-09-18T01:03:06.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318446+0000", 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318453+0000", 2024-09-18T01:03:06.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318458+0000", 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9910000000000002e-06 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346240+0000", 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027782305 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346316+0000", 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5731999999999999e-05 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346399+0000", 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3137e-05 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7859121690000004, 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:06.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:06.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7858916599999999, 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:06.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7858718549999999, 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7858372149999999, 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:06.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:06.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7858157139999999, 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:06.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:06.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7857925369999998, 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:06.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:06.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2237 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4202496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7818237520000002, 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028056793, 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2237 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:06.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322555+0000", 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7740000000000001e-06 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322562+0000", 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.426e-06 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322567+0000", 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350490+0000", 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027923436999999999 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350528+0000", 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7660999999999997e-05 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350610+0000", 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2150000000000005e-05 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7818037049999997, 2024-09-18T01:03:06.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:06.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:06.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7817841520000002, 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:06.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7817636480000001, 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:06.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7817427840000004, 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:06.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7817228839999997, 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.163 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:06.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7816987959999997, 2024-09-18T01:03:06.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:06.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.781667476, 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:06.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7816228699999996, 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7815920790000002, 2024-09-18T01:03:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7815685940000003, 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7815395389999997, 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7815187799999999, 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.200 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:06.201 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:06.200+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:06.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:06.203 INFO:teuthology.orchestra.run.smithi164.stdout:1497 write 0x7b0c000 thru 0x7b14376 (0x8377 bytes) 2024-09-18T01:03:06.203 INFO:teuthology.orchestra.run.smithi164.stdout:1498 read 0x21fa400 thru 0x2205616 (0xb217 bytes) 2024-09-18T01:03:06.203 INFO:teuthology.orchestra.run.smithi164.stdout:1499 read 0x7562e00 thru 0x7565362 (0x2563 bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1501 write 0x680e00 thru 0x68293c (0x1b3d bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1503 read 0xc59c00 thru 0xc65386 (0xb787 bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1504 write 0xc631800 thru 0xc63e6fc (0xcefd bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1505 write 0x8211800 thru 0x821bb96 (0xa397 bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1506 read 0x868f000 thru 0x8696e15 (0x7e16 bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1508 write 0x321ce00 thru 0x322689c (0x9a9d bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1509 punch from 0xc528400 to 0xc52e000, (0x5c00 bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1510 read 0xae3a200 thru 0xae3d0f7 (0x2ef8 bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1511 write 0x664ae00 thru 0x665795a (0xcb5b bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1512 write 0xb1e7c00 thru 0xb1f5547 (0xd948 bytes) 2024-09-18T01:03:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:1513 trunc from 0xc63e800 to 0x11bbc00 2024-09-18T01:03:06.302 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:06.301+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:06.403 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:06.402+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:06.404 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:06.405 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:06.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:06.423 INFO:teuthology.orchestra.run.smithi164.stdout:1516 write 0xeb1dc00 thru 0xeb2372e (0x5b2f bytes) 2024-09-18T01:03:06.424 INFO:teuthology.orchestra.run.smithi164.stdout:1518 write 0x16e9a00 thru 0x16f768c (0xdc8d bytes) 2024-09-18T01:03:06.425 INFO:teuthology.orchestra.run.smithi164.stdout:1519 write 0x3031c00 thru 0x303c59b (0xa99c bytes) 2024-09-18T01:03:06.425 INFO:teuthology.orchestra.run.smithi164.stdout:1522 punch from 0xd9dce00 to 0xd9e1800, (0x4a00 bytes) 2024-09-18T01:03:06.426 INFO:teuthology.orchestra.run.smithi164.stdout:1524 read 0xbd75400 thru 0xbd84f1e (0xfb1f bytes) 2024-09-18T01:03:06.426 INFO:teuthology.orchestra.run.smithi164.stdout:1525 read 0x12bea00 thru 0x12cadaf (0xc3b0 bytes) 2024-09-18T01:03:06.427 INFO:teuthology.orchestra.run.smithi164.stdout:1527 punch from 0xd78b800 to 0xd78bc00, (0x400 bytes) 2024-09-18T01:03:06.427 INFO:teuthology.orchestra.run.smithi164.stdout:1532 clone 9 order 21 su 32768 sc 15 2024-09-18T01:03:06.504 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:06.503+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:06.561 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:06.561 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:06.562 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:06.562 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:06.562 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:06.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:06.614 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:06.613+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.399607121999999, 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.394981219000002, 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.394903169999999, 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1292 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1774592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3381314450000001, 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050913359999999998, 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1292 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338601+0000", 2024-09-18T01:03:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.024e-06 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338610+0000", 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5469999999999997e-06 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338616+0000", 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389393+0000", 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05077661 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389472+0000", 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9776000000000003e-05 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389512+0000", 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0000000000000003e-05 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3381071270000007, 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3380812629999994, 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3380587829999993, 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3380377469999996, 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3380103529999996, 2024-09-18T01:03:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3379860479999994, 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3345865650000004, 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3345620050000004, 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3345204880000008, 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3344999860000009, 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3344795200000004, 2024-09-18T01:03:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3344381799999994, 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3344181420000005, 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3343758109999992, 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3343548189999996, 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3343038489999994, 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:06.777 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:06.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:06.779 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:06.777+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:06.880 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:06.879+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:06.926 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:06.926 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:06.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:06.982 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:06.981+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:07.045 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:07.046 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:07.046 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:07.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:07.095 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:07.094+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:07.183 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:07.181+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4670322179999999, 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4670122599999997, 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4669933190000002, 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4669718100000004, 2024-09-18T01:03:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4669460819999998, 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4582142229999997, 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4581915089999997, 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4581723789999996, 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4581535329999999, 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4581346550000003, 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4581127900000004, 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4580891549999997, 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4580555049999999, 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.45802765, 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4580078270000003, 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4579873929999998, 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4579653129999999, 2024-09-18T01:03:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4579429910000004, 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.457920595, 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4579001759999999, 2024-09-18T01:03:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:07.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:07.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:07.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:07.331 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:07.333 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:07.333 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:07.331+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:07.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:03:07.433 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:07.432+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:07.456 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:07.457 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:07.457 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:07.457 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:07.463 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone8 intact 2024-09-18T01:03:07.466 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:07.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:07.551 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:07.550+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:07.595 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:07.595 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:07.612 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:07.612 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:07.612 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:07.612 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:07.612 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:07.613 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:07.613 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:07.621 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:07.629 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:07.630 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:07.634 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:07.634+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:07.646 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:07.647 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:07.665 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:07.682 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:07.682 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:07.735 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:07.734+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:07.769 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:07.769 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:07.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:07.862 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:07.861+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:07.880 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-18T01:03:07.936 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:07.936 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:07.937 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:07.936+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:07.948 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:08.037 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:08.036+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:08.138 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:08.137+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:08.143 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:08.144 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:08.155 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:08.239 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:08.238+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:08.327 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:08.328 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:08.338 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:08.379 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:08.378+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:08.389 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:03:08.441 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:08.439+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:08.509 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:08.510 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:08.520 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:08.561 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:08.560+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:08.643 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:08.642+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:08.689 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:08.689 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:08.698 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:08.744 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:08.743+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:08.804 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:08.805 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:08.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:08.853 INFO:teuthology.orchestra.run.smithi164.stdout:1535 write 0x573b000 thru 0x5742001 (0x7002 bytes) 2024-09-18T01:03:08.854 INFO:teuthology.orchestra.run.smithi164.stdout:1536 punch from 0x9867600 to 0x9871400, (0x9e00 bytes) 2024-09-18T01:03:08.857 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:08.856+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:08.868 INFO:teuthology.orchestra.run.smithi164.stdout:1537 read 0x35a1800 thru 0x35a314e (0x194f bytes) 2024-09-18T01:03:08.868 INFO:teuthology.orchestra.run.smithi164.stdout:1538 read 0x1383c00 thru 0x1389935 (0x5d36 bytes) 2024-09-18T01:03:08.870 INFO:teuthology.orchestra.run.smithi164.stdout:1542 trunc from 0xeb23800 to 0xd0a2e00 2024-09-18T01:03:08.904 INFO:teuthology.orchestra.run.smithi164.stdout:1545 read 0xa3ac000 thru 0xa3acc12 (0xc13 bytes) 2024-09-18T01:03:08.905 INFO:teuthology.orchestra.run.smithi164.stdout:1548 write 0x1cce000 thru 0x1cddb04 (0xfb05 bytes) 2024-09-18T01:03:08.905 INFO:teuthology.orchestra.run.smithi164.stdout:1549 read 0x382ca00 thru 0x3836d13 (0xa314 bytes) 2024-09-18T01:03:08.909 INFO:teuthology.orchestra.run.smithi164.stdout:1551 trunc from 0xd0a2e00 to 0x9f4ec00 2024-09-18T01:03:08.938 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:08.938 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:08.946 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:08.944+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:08.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:08.962 INFO:teuthology.orchestra.run.smithi164.stdout:1553 read 0x3fc6c00 thru 0x3fc93d7 (0x27d8 bytes) 2024-09-18T01:03:08.966 INFO:teuthology.orchestra.run.smithi164.stdout:1555 trunc from 0x9f4ec00 to 0xbb93200 2024-09-18T01:03:08.973 INFO:teuthology.orchestra.run.smithi164.stdout:1557 punch from 0x4cbf800 to 0x4ccce00, (0xd600 bytes) 2024-09-18T01:03:08.976 INFO:teuthology.orchestra.run.smithi164.stdout:1558 read 0x2ebe800 thru 0x2ec9214 (0xaa15 bytes) 2024-09-18T01:03:08.976 INFO:teuthology.orchestra.run.smithi164.stdout:1559 write 0x6d54400 thru 0x6d56d90 (0x2991 bytes) 2024-09-18T01:03:08.976 INFO:teuthology.orchestra.run.smithi164.stdout:1560 read 0xb5fc200 thru 0xb606c5f (0xaa60 bytes) 2024-09-18T01:03:08.976 INFO:teuthology.orchestra.run.smithi164.stdout:1561 write 0x8bad800 thru 0x8bbceff (0xf700 bytes) 2024-09-18T01:03:08.984 INFO:teuthology.orchestra.run.smithi164.stdout:1562 trunc from 0xbb93200 to 0x4bd1c00 2024-09-18T01:03:09.042 INFO:teuthology.orchestra.run.smithi164.stdout:1563 trunc from 0x4bd1c00 to 0xe565600 2024-09-18T01:03:09.047 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:09.046+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:09.051 INFO:teuthology.orchestra.run.smithi164.stdout:1569 punch from 0x2204000 to 0x2209200, (0x5200 bytes) 2024-09-18T01:03:09.053 INFO:teuthology.orchestra.run.smithi164.stdout:1572 read 0x25a7e00 thru 0x25b1092 (0x9293 bytes) 2024-09-18T01:03:09.053 INFO:teuthology.orchestra.run.smithi164.stdout:1576 write 0x5e46400 thru 0x5e4d84e (0x744f bytes) 2024-09-18T01:03:09.054 INFO:teuthology.orchestra.run.smithi164.stdout:1577 read 0xa481e00 thru 0xa48729f (0x54a0 bytes) 2024-09-18T01:03:09.067 INFO:teuthology.orchestra.run.smithi164.stdout:1578 trunc from 0xe565600 to 0x2fe9600 2024-09-18T01:03:09.103 INFO:teuthology.orchestra.run.smithi164.stdout:1580 read 0x1605e00 thru 0x161189b (0xba9c bytes) 2024-09-18T01:03:09.105 INFO:teuthology.orchestra.run.smithi164.stdout:1584 read 0x2cef400 thru 0x2cefcdc (0x8dd bytes) 2024-09-18T01:03:09.110 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:09.111 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:09.114 INFO:teuthology.orchestra.run.smithi164.stdout:1585 trunc from 0x2fe9600 to 0xa6d7800 2024-09-18T01:03:09.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:09.122 INFO:teuthology.orchestra.run.smithi164.stdout:1586 read 0x6e97e00 thru 0x6ea33d1 (0xb5d2 bytes) 2024-09-18T01:03:09.127 INFO:teuthology.orchestra.run.smithi164.stdout:1589 trunc from 0xa6d7800 to 0xdbd3800 2024-09-18T01:03:09.143 INFO:teuthology.orchestra.run.smithi164.stdout:1590 trunc from 0xdbd3800 to 0x80df800 2024-09-18T01:03:09.149 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:09.148+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:09.153 INFO:teuthology.orchestra.run.smithi164.stdout:1592 punch from 0x6ce200 to 0x6d5400, (0x7200 bytes) 2024-09-18T01:03:09.159 INFO:teuthology.orchestra.run.smithi164.stdout:1593 trunc from 0x80df800 to 0xc70a000 2024-09-18T01:03:09.174 INFO:teuthology.orchestra.run.smithi164.stdout:1595 trunc from 0xc70a000 to 0x87c4000 2024-09-18T01:03:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:1597 write 0xb601600 thru 0xb60e7d2 (0xd1d3 bytes) 2024-09-18T01:03:09.197 INFO:teuthology.orchestra.run.smithi164.stdout:1598 write 0x6f1c00 thru 0x700d0d (0xf10e bytes) 2024-09-18T01:03:09.198 INFO:teuthology.orchestra.run.smithi164.stdout:1599 punch from 0x2538800 to 0x2544c00, (0xc400 bytes) 2024-09-18T01:03:09.202 INFO:teuthology.orchestra.run.smithi164.stdout:1600 trunc from 0xb60e800 to 0x9a93600 2024-09-18T01:03:09.237 INFO:teuthology.orchestra.run.smithi164.stdout:1604 write 0x5ea5600 thru 0x5eaec5f (0x9660 bytes) 2024-09-18T01:03:09.237 INFO:teuthology.orchestra.run.smithi164.stdout:1605 punch from 0x2621600 to 0x2623c00, (0x2600 bytes) 2024-09-18T01:03:09.237 INFO:teuthology.orchestra.run.smithi164.stdout:1611 read 0x8f8a00 thru 0x903a58 (0xb059 bytes) 2024-09-18T01:03:09.238 INFO:teuthology.orchestra.run.smithi164.stdout:1612 read 0x8da000 thru 0x8dcc87 (0x2c88 bytes) 2024-09-18T01:03:09.239 INFO:teuthology.orchestra.run.smithi164.stdout:1614 read 0x2ed1c00 thru 0x2edff42 (0xe343 bytes) 2024-09-18T01:03:09.239 INFO:teuthology.orchestra.run.smithi164.stdout:1615 write 0x5abdc00 thru 0x5ac61f6 (0x85f7 bytes) 2024-09-18T01:03:09.239 INFO:teuthology.orchestra.run.smithi164.stdout:1617 write 0x5627000 thru 0x5632098 (0xb099 bytes) 2024-09-18T01:03:09.250 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:09.249+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:09.258 INFO:teuthology.orchestra.run.smithi164.stdout:1619 write 0x9bcfc00 thru 0x9bd556e (0x596f bytes) 2024-09-18T01:03:09.259 INFO:teuthology.orchestra.run.smithi164.stdout:1623 read 0x46c5c00 thru 0x46cf889 (0x9c8a bytes) 2024-09-18T01:03:09.259 INFO:teuthology.orchestra.run.smithi164.stdout:1624 write 0x32f9a00 thru 0x3303f57 (0xa558 bytes) 2024-09-18T01:03:09.260 INFO:teuthology.orchestra.run.smithi164.stdout:1625 punch from 0x309dc00 to 0x30ac600, (0xea00 bytes) 2024-09-18T01:03:09.277 INFO:teuthology.orchestra.run.smithi164.stdout:1626 write 0xa38b200 thru 0xa38e6ea (0x34eb bytes) 2024-09-18T01:03:09.277 INFO:teuthology.orchestra.run.smithi164.stdout:1627 read 0x1f07200 thru 0x1f1679e (0xf59f bytes) 2024-09-18T01:03:09.278 INFO:teuthology.orchestra.run.smithi164.stdout:1629 write 0x1a43c00 thru 0x1a520c1 (0xe4c2 bytes) 2024-09-18T01:03:09.278 INFO:teuthology.orchestra.run.smithi164.stdout:1631 read 0x8ef4000 thru 0x8ef5a0f (0x1a10 bytes) 2024-09-18T01:03:09.278 INFO:teuthology.orchestra.run.smithi164.stdout:1633 read 0x9ab2800 thru 0x9ac254e (0xfd4f bytes) 2024-09-18T01:03:09.279 INFO:teuthology.orchestra.run.smithi164.stdout:1638 write 0x999d000 thru 0x99a32c5 (0x62c6 bytes) 2024-09-18T01:03:09.279 INFO:teuthology.orchestra.run.smithi164.stdout:1639 read 0x77f2200 thru 0x77fc2cc (0xa0cd bytes) 2024-09-18T01:03:09.279 INFO:teuthology.orchestra.run.smithi164.stdout:1640 read 0xb69400 thru 0xb73025 (0x9c26 bytes) 2024-09-18T01:03:09.279 INFO:teuthology.orchestra.run.smithi164.stdout:1642 read 0x2fd8a00 thru 0x2fe7cc7 (0xf2c8 bytes) 2024-09-18T01:03:09.283 INFO:teuthology.orchestra.run.smithi164.stdout:1647 trunc from 0xa38e800 to 0xd161e00 2024-09-18T01:03:09.284 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:09.284 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:09.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:09.310 INFO:teuthology.orchestra.run.smithi164.stdout:1648 read 0xd085800 thru 0xd08b9d8 (0x61d9 bytes) 2024-09-18T01:03:09.311 INFO:teuthology.orchestra.run.smithi164.stdout:1650 write 0x9fbba00 thru 0x9fcb22b (0xf82c bytes) 2024-09-18T01:03:09.321 INFO:teuthology.orchestra.run.smithi164.stdout:1651 trunc from 0xd161e00 to 0x4fc6800 2024-09-18T01:03:09.343 INFO:teuthology.orchestra.run.smithi164.stdout:1653 clone 10 order 25 su 2097152 sc 8 2024-09-18T01:03:09.353 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:09.352+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:09.416 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:09.416 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:09.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:09.466 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:09.465+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:09.468 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone9 from 0x4fc6800 (overlap 0x2fe9600) to 0x0 2024-09-18T01:03:09.558 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:09.557+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:09.584 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:09.584 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:09.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:09.660 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:09.658+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:09.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:09.702 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:09.703 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:09.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:09.761 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:09.760+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:09.823 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:09.823 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:09.833 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-18T01:03:09.842 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:09.842 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:09.859 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:09.860 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:09.862 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:09.861+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:09.867 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:09.867 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:09.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:09.876 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:09.877 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:09.898 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:09.898 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:09.925 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:09.925 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:09.981 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:09.980+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:10.012 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:10.012 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:10.012 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:10.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:10.065 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:10.064+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:10.166 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:10.165+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4575321139999993, 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:10.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:10.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:10.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4575121559999999, 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:10.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:10.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:10.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:10.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.208 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4574932149999995, 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:10.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:10.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:10.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4574717059999998, 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4574459779999991, 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:10.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:10.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4487141189999999, 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4486914049999999, 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:10.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:10.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4486722749999998, 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:10.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:10.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:10.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4486534290000002, 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:10.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:10.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4486345509999996, 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:10.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4486126860000006, 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4485890510000008, 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:10.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:10.272 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4485554010000001, 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:10.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:10.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4485275459999993, 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:10.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:10.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:10.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4485077230000005, 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:10.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:10.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4484872889999991, 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:10.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:10.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:10.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4484652090000001, 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4484428870000006, 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:10.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:10.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4484204910000003, 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4484000720000001, 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.330 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:10.331 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:10.332 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:10.330+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:10.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:03:10.422 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:10.422 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:10.422 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:03:10.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:03:10.434 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:10.435 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:10.435 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:10.435 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:10.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:10.451 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:10.450+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:10.532 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:10.531+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:10.547 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:10.547 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:10.547 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:10.547 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:10.547 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:10.547 INFO:teuthology.orchestra.run.smithi164.stdout:1654 write 0x57da800 thru 0x57e306a (0x886b bytes) 2024-09-18T01:03:10.557 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:10.576 INFO:teuthology.orchestra.run.smithi164.stdout:1655 write 0x89c2400 thru 0x89c993a (0x753b bytes) 2024-09-18T01:03:10.591 INFO:teuthology.orchestra.run.smithi164.stdout:1657 trunc from 0x89c9a00 to 0x1786a00 2024-09-18T01:03:10.642 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:10.641+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:10.699 INFO:teuthology.orchestra.run.smithi164.stdout:1659 read 0x177b200 thru 0x177b553 (0x354 bytes) 2024-09-18T01:03:10.700 INFO:teuthology.orchestra.run.smithi164.stdout:1660 punch from 0x17f200 to 0x186a00, (0x7800 bytes) 2024-09-18T01:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout:1661 punch from 0xd7a200 to 0xd84a00, (0xa800 bytes) 2024-09-18T01:03:10.704 INFO:teuthology.orchestra.run.smithi164.stdout:1664 read 0x16fe400 thru 0x1705d11 (0x7912 bytes) 2024-09-18T01:03:10.705 INFO:teuthology.orchestra.run.smithi164.stdout:1667 write 0x1523a00 thru 0x15291f1 (0x57f2 bytes) 2024-09-18T01:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout:1670 punch from 0x15a000 to 0x15be00, (0x1e00 bytes) 2024-09-18T01:03:10.706 INFO:teuthology.orchestra.run.smithi164.stdout:1672 write 0x136e600 thru 0x137d044 (0xea45 bytes) 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6693419380000005, 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.715 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:03:10.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:03:10.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6693307869999998, 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.720 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:03:10.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6693189569999998, 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:10.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6693022739999996, 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.730 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:03:10.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.734 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6692910370000007, 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:03:10.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:03:10.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:03:10.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6679689320000008, 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.740 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:03:10.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6679562170000004, 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.745 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6679112329999999, 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:10.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:03:10.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.754 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6678836090000004, 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:03:10.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:03:10.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:10.759 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.66433623, 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:03:10.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6639761229999994, 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.764 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:03:10.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.769 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6639500100000006, 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:03:10.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:03:10.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23940 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894162912] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.661780297, 2024-09-18T01:03:10.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037903539999999999, 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23940 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051180+0000", 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4190000000000001e-06 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051183+0000", 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.774e-06 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051185+0000", 2024-09-18T01:03:10.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5330000000000001e-06 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.088984+0000", 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037798867999999999 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089016+0000", 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2333999999999999e-05 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089082+0000", 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5399000000000002e-05 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6617706220000006, 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:03:10.779 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:03:10.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.783 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6617608270000002, 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.784 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:10.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:03:10.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:03:10.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.661750048, 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.791 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:03:10.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.795 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6617302519999999, 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.796 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:03:10.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6599467319999999, 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:03:10.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6599198939999997, 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.805 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:10.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:03:10.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 8.6599093459999992, 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:03:10.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:10.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:03:10.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:10.814 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:10.818 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 44, 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-18T01:02:13.493841+0000", 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-18T01:03:09.381406+0000", 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-18T01:02:24.290620+0000", 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-18T01:02:48.274416+0000", 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-18T01:03:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-18T01:03:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:24.388064+0000", 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-18T01:03:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-18T01:03:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-18T01:03:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:30.676199+0000", 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-18T01:03:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2699999809265137, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2699999809265137, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:40.390732+0000", 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "44", 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 12, 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 44, 2024-09-18T01:03:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:10.838 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:10.839 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.5099999904632568, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.5099999904632568, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.25, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.25, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:10.840 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4563fb16-8745-4e75-b654-1e96403d91b9", 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/1626166637", 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/1626166637", 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/1626166637", 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/1626166637", 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "634965cb-d444-4721-b34a-afbc49105cab", 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-18T01:03:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-18T01:03:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-18T01:03:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/2210970259", 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/2210970259", 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/2210970259", 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/2210970259", 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "83443eab-2fa4-4c7e-a459-edb1bb0ed8d0", 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/617133675", 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/617133675", 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/617133675", 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/617133675", 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "5797982a-71c1-4d6a-9604-00fbc6bd16d9", 2024-09-18T01:03:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6808", 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6809", 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6810", 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6811", 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6814", 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6815", 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6812", 2024-09-18T01:03:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6813", 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6809/151031464", 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6811/151031464", 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6815/151031464", 2024-09-18T01:03:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6813/151031464", 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f79e3329-5481-4f33-8d28-6386af186492", 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6816", 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6817", 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6818", 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6819", 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6822", 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6823", 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6820", 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6821", 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6817/1499036965", 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6819/1499036965", 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6823/1499036965", 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6821/1499036965", 2024-09-18T01:03:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "8c61d2ec-b111-4783-a016-336ed33a0694", 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 20, 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6800", 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6801", 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6802", 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6803", 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6806", 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6807", 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6804", 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6805", 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6801/2310498524", 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6803/2310498524", 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6807/2310498524", 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6805/2310498524", 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.877700+0000", 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.360930+0000", 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.403205+0000", 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.952123+0000", 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:22.099545+0000", 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.694132+0000", 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-18T01:03:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-18T01:03:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-18T01:03:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-18T01:03:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:03:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:03:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-18T01:03:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:10.883 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:10.884 INFO:tasks.thrashosds.thrasher:j is {'epoch': 44, 'fsid': '62b46b83-8ef2-466f-b28b-2d97f265422e', 'created': '2024-09-18T01:02:13.493841+0000', 'modified': '2024-09-18T01:03:09.381406+0000', 'last_up_change': '2024-09-18T01:02:24.290620+0000', 'last_in_change': '2024-09-18T01:02:48.274416+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:02:24.388064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:02:30.676199+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:02:40.390732+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 12, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4563fb16-8745-4e75-b654-1e96403d91b9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 1626166637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 1626166637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 1626166637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 1626166637}]}, 'public_addr': '172.21.15.31:6817/1626166637', 'cluster_addr': '172.21.15.31:6819/1626166637', 'heartbeat_back_addr': '172.21.15.31:6823/1626166637', 'heartbeat_front_addr': '172.21.15.31:6821/1626166637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '634965cb-d444-4721-b34a-afbc49105cab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 2210970259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 2210970259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 2210970259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 2210970259}]}, 'public_addr': '172.21.15.31:6801/2210970259', 'cluster_addr': '172.21.15.31:6803/2210970259', 'heartbeat_back_addr': '172.21.15.31:6807/2210970259', 'heartbeat_front_addr': '172.21.15.31:6805/2210970259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 617133675}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 617133675}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 617133675}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 617133675}]}, 'public_addr': '172.21.15.31:6809/617133675', 'cluster_addr': '172.21.15.31:6811/617133675', 'heartbeat_back_addr': '172.21.15.31:6815/617133675', 'heartbeat_front_addr': '172.21.15.31:6813/617133675', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6808', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6809', 'nonce': 151031464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6810', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6811', 'nonce': 151031464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6814', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6815', 'nonce': 151031464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6812', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6813', 'nonce': 151031464}]}, 'public_addr': '172.21.15.98:6809/151031464', 'cluster_addr': '172.21.15.98:6811/151031464', 'heartbeat_back_addr': '172.21.15.98:6815/151031464', 'heartbeat_front_addr': '172.21.15.98:6813/151031464', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f79e3329-5481-4f33-8d28-6386af186492', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6816', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6817', 'nonce': 1499036965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6818', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6819', 'nonce': 1499036965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6822', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6823', 'nonce': 1499036965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6820', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6821', 'nonce': 1499036965}]}, 'public_addr': '172.21.15.98:6817/1499036965', 'cluster_addr': '172.21.15.98:6819/1499036965', 'heartbeat_back_addr': '172.21.15.98:6823/1499036965', 'heartbeat_front_addr': '172.21.15.98:6821/1499036965', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8c61d2ec-b111-4783-a016-336ed33a0694', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6800', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6801', 'nonce': 2310498524}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6802', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6803', 'nonce': 2310498524}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6806', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6807', 'nonce': 2310498524}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6804', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6805', 'nonce': 2310498524}]}, 'public_addr': '172.21.15.98:6801/2310498524', 'cluster_addr': '172.21.15.98:6803/2310498524', 'heartbeat_back_addr': '172.21.15.98:6807/2310498524', 'heartbeat_front_addr': '172.21.15.98:6805/2310498524', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.877700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.360930+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.403205+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.952123+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:22.099545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.694132+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:03:10.884 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T01:03:10.884 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:10.815+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:10.884 INFO:teuthology.orchestra.run.smithi164.stdout:1673 write 0x9820800 thru 0x9822c44 (0x2445 bytes) 2024-09-18T01:03:10.884 INFO:teuthology.orchestra.run.smithi164.stdout:1674 write 0xac6ce00 thru 0xac7b15b (0xe35c bytes) 2024-09-18T01:03:10.884 INFO:teuthology.orchestra.run.smithi164.stdout:1675 read 0x41b0600 thru 0x41b612d (0x5b2e bytes) 2024-09-18T01:03:10.884 INFO:teuthology.orchestra.run.smithi164.stdout:1677 write 0x763f200 thru 0x7641bc3 (0x29c4 bytes) 2024-09-18T01:03:10.885 INFO:teuthology.orchestra.run.smithi164.stdout:1684 write 0xe56d400 thru 0xe56fa25 (0x2626 bytes) 2024-09-18T01:03:10.885 INFO:teuthology.orchestra.run.smithi164.stdout:1685 read 0x7838200 thru 0x7846b9f (0xe9a0 bytes) 2024-09-18T01:03:10.885 INFO:teuthology.orchestra.run.smithi164.stdout:1686 write 0xa981400 thru 0xa9887bb (0x73bc bytes) 2024-09-18T01:03:10.885 INFO:teuthology.orchestra.run.smithi164.stdout:1687 clone 11 order 23 su 32768 sc 10 2024-09-18T01:03:10.928 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:10.927+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:10.987 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:10.987 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:10.987 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:10.988 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:10.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:11.018 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:11.017+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:11.119 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:11.118+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:11.140 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:11.140 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:11.140 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:11.140 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:11.140 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:11.148 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:11.221 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:11.220+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:11.250 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:11.250 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:11.250 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:11.250 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9528071919999999, 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:03:11.251 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.252 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:03:11.253 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.254 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9527830750000001, 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.255 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.256 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:03:11.257 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.258 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.259 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9527038350000003, 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.260 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:03:11.261 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:03:11.262 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.263 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9526676160000003, 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.264 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.265 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.266 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.267 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9526390029999998, 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:03:11.268 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.269 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:03:11.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:03:11.271 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9526181380000001, 2024-09-18T01:03:11.272 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:11.273 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:03:11.274 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.275 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:03:11.276 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9520004210000002, 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.277 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:11.278 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.279 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.280 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9519711539999998, 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:03:11.281 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:03:11.283 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.284 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.285 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9519048630000002, 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.286 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.287 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.288 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.289 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9517840959999999, 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.290 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.292 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:11.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.294 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.95156565, 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.295 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.297 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:03:11.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9515442890000001, 2024-09-18T01:03:11.299 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:11.300 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:11.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.303 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9513903500000005, 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.304 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:03:11.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9513706319999997, 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.308 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.312 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9513503219999997, 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.313 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:03:11.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9513304360000001, 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:03:11.317 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:03:11.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.321 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9513104769999998, 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.322 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:03:11.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:03:11.325 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9512904789999999, 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.326 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.327 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:03:11.328 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.330 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9512708119999997, 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.331 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.332 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:03:11.333 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.9512509099999997, 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.335 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.336 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.337 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.339 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.340 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:11.341 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:11.342 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:11.340+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:03:11.451 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:11.452 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:11.452 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:11.452 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:11.457 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:11.456+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:11.461 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:11.503 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone10 from 0xe56fc00 (overlap 0x1786a00) to 0x0 2024-09-18T01:03:11.542 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:11.541+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:11.571 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:11.571 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:11.571 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:11.571 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:11.571 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:11.580 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:11.643 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:11.642+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:11.686 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:11.686 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:11.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:11.686 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.310473674000001, 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:11.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:11.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:11.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:11.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 38.312462369000002, 2024-09-18T01:03:11.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:11.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.696 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:11.699 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 37.304785416999998, 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:11.700 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:11.706 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 28.206927467, 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:11.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:11.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.350846454999999, 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.714 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:11.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:11.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.350477159, 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:11.720 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:11.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:11.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.348738313999998, 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:11.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:11.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.348666424000001, 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:11.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.260800413999998, 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:11.736 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:11.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:11.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.254609155000001, 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.742 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:11.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.253079469999999, 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.747 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:11.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:11.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.252699781, 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:11.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:11.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:11.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.237201378000002, 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.759 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:11.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:11.765 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.236994619000001, 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:11.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:11.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:11.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:11.773 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.236576862, 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:11.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:11.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.236367624, 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.778 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:11.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.782 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.23630369, 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 24.235555484999999, 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:11.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.976616679999999, 2024-09-18T01:03:11.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:11.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:11.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:11.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.798 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.975753061999999, 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:11.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:11.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:11.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:11.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:11.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:11.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:11.805 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:11.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:11.808 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:11.806+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:11.907 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:11.906+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:11.925 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:11.926 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:11.926 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:11.926 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:11.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:11.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:12.009 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:12.008+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:12.044 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-18T01:03:12.088 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:12.089 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:12.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:12.100 INFO:teuthology.orchestra.run.smithi164.stdout:1688 read 0xab9000 thru 0xac7b20 (0xeb21 bytes) 2024-09-18T01:03:12.101 INFO:teuthology.orchestra.run.smithi164.stdout:1691 read 0xa43c400 thru 0xa443a40 (0x7641 bytes) 2024-09-18T01:03:12.102 INFO:teuthology.orchestra.run.smithi164.stdout:1692 punch from 0x96a5c00 to 0x96ab800, (0x5c00 bytes) 2024-09-18T01:03:12.104 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:12.104 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:12.109 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:12.108+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:12.133 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:12.154 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:12.154 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:12.155 INFO:teuthology.orchestra.run.smithi164.stdout:1693 trunc from 0xe56fc00 to 0x78bda00 2024-09-18T01:03:12.192 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:12.192 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:12.211 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:12.209+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:12.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:12.238 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:12.258 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:12.258 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:12.265 INFO:teuthology.orchestra.run.smithi164.stdout:1697 read 0x4605e00 thru 0x4607c93 (0x1e94 bytes) 2024-09-18T01:03:12.265 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:12.266 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:12.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:12.274 INFO:teuthology.orchestra.run.smithi164.stdout:1698 trunc from 0x78bda00 to 0x8628600 2024-09-18T01:03:12.284 INFO:teuthology.orchestra.run.smithi164.stdout:1701 trunc from 0x8628600 to 0x7fe9800 2024-09-18T01:03:12.304 INFO:teuthology.orchestra.run.smithi164.stdout:1702 punch from 0x34cfc00 to 0x34d2800, (0x2c00 bytes) 2024-09-18T01:03:12.311 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:12.310+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:12.339 INFO:teuthology.orchestra.run.smithi164.stdout:1703 write 0xe597200 thru 0xe5a60e9 (0xeeea bytes) 2024-09-18T01:03:12.340 INFO:teuthology.orchestra.run.smithi164.stdout:1704 read 0x9d9c000 thru 0x9da1e87 (0x5e88 bytes) 2024-09-18T01:03:12.355 INFO:teuthology.orchestra.run.smithi164.stdout:1706 trunc from 0xe5a6200 to 0x2e95400 2024-09-18T01:03:12.413 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:12.410+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:12.479 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:12.480 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:12.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:12.508 INFO:teuthology.orchestra.run.smithi164.stdout:1707 write 0x8829600 thru 0x8839515 (0xff16 bytes) 2024-09-18T01:03:12.531 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:12.530+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:12.533 INFO:teuthology.orchestra.run.smithi164.stdout:1708 write 0x8dabc00 thru 0x8db1d27 (0x6128 bytes) 2024-09-18T01:03:12.535 INFO:teuthology.orchestra.run.smithi164.stdout:1711 trunc from 0x8db1e00 to 0x6c21a00 2024-09-18T01:03:12.561 INFO:teuthology.orchestra.run.smithi164.stdout:1712 read 0x2b1b400 thru 0x2b29945 (0xe546 bytes) 2024-09-18T01:03:12.562 INFO:teuthology.orchestra.run.smithi164.stdout:1713 punch from 0x34d1a00 to 0x34da200, (0x8800 bytes) 2024-09-18T01:03:12.562 INFO:teuthology.orchestra.run.smithi164.stdout:1715 write 0x2d01000 thru 0x2d084fe (0x74ff bytes) 2024-09-18T01:03:12.563 INFO:teuthology.orchestra.run.smithi164.stdout:1716 write 0x62e3600 thru 0x62eb8a0 (0x82a1 bytes) 2024-09-18T01:03:12.564 INFO:teuthology.orchestra.run.smithi164.stdout:1724 punch from 0x28cd400 to 0x28d8c00, (0xb800 bytes) 2024-09-18T01:03:12.580 INFO:teuthology.orchestra.run.smithi164.stdout:1726 write 0xc3a3600 thru 0xc3a7957 (0x4358 bytes) 2024-09-18T01:03:12.587 INFO:teuthology.orchestra.run.smithi164.stdout:1732 trunc from 0xc3a7a00 to 0xa450c00 2024-09-18T01:03:12.601 INFO:teuthology.orchestra.run.smithi164.stdout:1733 write 0x46c3400 thru 0x46c84c1 (0x50c2 bytes) 2024-09-18T01:03:12.602 INFO:teuthology.orchestra.run.smithi164.stdout:1734 read 0x4ccc200 thru 0x4cce36c (0x216d bytes) 2024-09-18T01:03:12.613 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:12.612+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:12.614 INFO:teuthology.orchestra.run.smithi164.stdout:1736 write 0xb840000 thru 0xb84b510 (0xb511 bytes) 2024-09-18T01:03:12.615 INFO:teuthology.orchestra.run.smithi164.stdout:1738 punch from 0xa64bc00 to 0xa654600, (0x8a00 bytes) 2024-09-18T01:03:12.615 INFO:teuthology.orchestra.run.smithi164.stdout:1739 punch from 0xa6fbe00 to 0xa700a00, (0x4c00 bytes) 2024-09-18T01:03:12.616 INFO:teuthology.orchestra.run.smithi164.stdout:1740 read 0x8c5b200 thru 0x8c68e52 (0xdc53 bytes) 2024-09-18T01:03:12.624 INFO:teuthology.orchestra.run.smithi164.stdout:1744 trunc from 0xb84b600 to 0x50ad400 2024-09-18T01:03:12.640 INFO:teuthology.orchestra.run.smithi164.stdout:1747 punch from 0x480ac00 to 0x4811800, (0x6c00 bytes) 2024-09-18T01:03:12.649 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:12.649 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:12.651 INFO:teuthology.orchestra.run.smithi164.stdout:1748 trunc from 0x50ad400 to 0xd319200 2024-09-18T01:03:12.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:12.666 INFO:teuthology.orchestra.run.smithi164.stdout:1750 read 0xd127600 thru 0xd12c6d6 (0x50d7 bytes) 2024-09-18T01:03:12.667 INFO:teuthology.orchestra.run.smithi164.stdout:1752 read 0x32a1600 thru 0x32afa86 (0xe487 bytes) 2024-09-18T01:03:12.667 INFO:teuthology.orchestra.run.smithi164.stdout:1753 write 0x7491000 thru 0x749f3bc (0xe3bd bytes) 2024-09-18T01:03:12.667 INFO:teuthology.orchestra.run.smithi164.stdout:1755 read 0x2500200 thru 0x250e8e8 (0xe6e9 bytes) 2024-09-18T01:03:12.667 INFO:teuthology.orchestra.run.smithi164.stdout:1757 write 0x9f5000 thru 0x9f6701 (0x1702 bytes) 2024-09-18T01:03:12.668 INFO:teuthology.orchestra.run.smithi164.stdout:1758 read 0x5bae800 thru 0x5bb29ff (0x4200 bytes) 2024-09-18T01:03:12.668 INFO:teuthology.orchestra.run.smithi164.stdout:1763 read 0x6208c00 thru 0x620fb1d (0x6f1e bytes) 2024-09-18T01:03:12.668 INFO:teuthology.orchestra.run.smithi164.stdout:1765 write 0x11e7a00 thru 0x11e80f5 (0x6f6 bytes) 2024-09-18T01:03:12.670 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xedd5c00 2024-09-18T01:03:12.672 INFO:teuthology.orchestra.run.smithi164.stdout:1766 trunc from 0xd319200 to 0xedd5c00 2024-09-18T01:03:12.697 INFO:teuthology.orchestra.run.smithi164.stdout:1768 trunc from 0xedd5c00 to 0x3bddc00 2024-09-18T01:03:12.714 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:12.713+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:12.714 INFO:teuthology.orchestra.run.smithi164.stdout:1770 write 0x2105000 thru 0x210b396 (0x6397 bytes) 2024-09-18T01:03:12.715 INFO:teuthology.orchestra.run.smithi164.stdout:1773 read 0xc6d200 thru 0xc7823e (0xb03f bytes) 2024-09-18T01:03:12.716 INFO:teuthology.orchestra.run.smithi164.stdout:1776 punch from 0x21aa400 to 0x21b7800, (0xd400 bytes) 2024-09-18T01:03:12.717 INFO:teuthology.orchestra.run.smithi164.stdout:1777 punch from 0x700400 to 0x70ca00, (0xc600 bytes) 2024-09-18T01:03:12.717 INFO:teuthology.orchestra.run.smithi164.stdout:1778 punch from 0x6b4400 to 0x6bf000, (0xac00 bytes) 2024-09-18T01:03:12.717 INFO:teuthology.orchestra.run.smithi164.stdout:1782 write 0x172e600 thru 0x1732d29 (0x472a bytes) 2024-09-18T01:03:12.718 INFO:teuthology.orchestra.run.smithi164.stdout:1783 read 0x365a00 thru 0x37479e (0xed9f bytes) 2024-09-18T01:03:12.732 INFO:teuthology.orchestra.run.smithi164.stdout:1784 trunc from 0x3bddc00 to 0xe4e2400 2024-09-18T01:03:12.744 INFO:teuthology.orchestra.run.smithi164.stdout:1785 punch from 0x4f6c00 to 0x4fd200, (0x6600 bytes) 2024-09-18T01:03:12.745 INFO:teuthology.orchestra.run.smithi164.stdout:1786 write 0xce75a00 thru 0xce83233 (0xd834 bytes) 2024-09-18T01:03:12.745 INFO:teuthology.orchestra.run.smithi164.stdout:1788 read 0xfbf400 thru 0xfc8fc3 (0x9bc4 bytes) 2024-09-18T01:03:12.746 INFO:teuthology.orchestra.run.smithi164.stdout:1789 read 0x5164200 thru 0x5165021 (0xe22 bytes) 2024-09-18T01:03:12.746 INFO:teuthology.orchestra.run.smithi164.stdout:1791 read 0xcc1d200 thru 0xcc25c4a (0x8a4b bytes) 2024-09-18T01:03:12.746 INFO:teuthology.orchestra.run.smithi164.stdout:1792 write 0x364bc00 thru 0x364db53 (0x1f54 bytes) 2024-09-18T01:03:12.746 INFO:teuthology.orchestra.run.smithi164.stdout:1793 write 0xb120800 thru 0xb12fd93 (0xf594 bytes) 2024-09-18T01:03:12.748 INFO:teuthology.orchestra.run.smithi164.stdout:1794 trunc from 0xe4e2400 to 0xcb99600 2024-09-18T01:03:12.766 INFO:teuthology.orchestra.run.smithi164.stdout:1795 read 0xa5bfe00 thru 0xa5c50ed (0x52ee bytes) 2024-09-18T01:03:12.767 INFO:teuthology.orchestra.run.smithi164.stdout:1796 read 0x3370800 thru 0x3371068 (0x869 bytes) 2024-09-18T01:03:12.768 INFO:teuthology.orchestra.run.smithi164.stdout:1802 read 0xbb0f400 thru 0xbb0fb4e (0x74f bytes) 2024-09-18T01:03:12.768 INFO:teuthology.orchestra.run.smithi164.stdout:1803 write 0x826fc00 thru 0x827bdf4 (0xc1f5 bytes) 2024-09-18T01:03:12.768 INFO:teuthology.orchestra.run.smithi164.stdout:1804 write 0x7e02e00 thru 0x7e07ac1 (0x4cc2 bytes) 2024-09-18T01:03:12.768 INFO:teuthology.orchestra.run.smithi164.stdout:1805 write 0x7456400 thru 0x745a24e (0x3e4f bytes) 2024-09-18T01:03:12.768 INFO:teuthology.orchestra.run.smithi164.stdout:1807 read 0x235c00 thru 0x239852 (0x3c53 bytes) 2024-09-18T01:03:12.769 INFO:teuthology.orchestra.run.smithi164.stdout:1814 read 0x5ebb000 thru 0x5eca73c (0xf73d bytes) 2024-09-18T01:03:12.769 INFO:teuthology.orchestra.run.smithi164.stdout:1816 read 0x4ba3800 thru 0x4bacab1 (0x92b2 bytes) 2024-09-18T01:03:12.769 INFO:teuthology.orchestra.run.smithi164.stdout:1817 write 0x4520a00 thru 0x45257b0 (0x4db1 bytes) 2024-09-18T01:03:12.770 INFO:teuthology.orchestra.run.smithi164.stdout:1820 write 0x24d7400 thru 0x24d878f (0x1390 bytes) 2024-09-18T01:03:12.770 INFO:teuthology.orchestra.run.smithi164.stdout:1821 clone 12 order 20 su 8192 sc 14 2024-09-18T01:03:12.782 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:12.782 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:12.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:12.835 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:12.834+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:12.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:12.915 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:12.914+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:13.017 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:13.016+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:13.053 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:13.054 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:13.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:13.117 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:13.116+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:13.169 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:13.169 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:13.178 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:13.222 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:13.221+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:13.284 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:13.284 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:13.292 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:13.319 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:13.318+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:13.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:03:13.420 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:13.419+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:13.440 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:13.441 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:13.450 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:13.521 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:13.520+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:13.529 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone11 from 0xcb99600 (overlap 0x2e95400) to 0x0 2024-09-18T01:03:13.561 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:13.561 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:13.569 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:13.655 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:13.654+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:13.716 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:13.716 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:13.723 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:13.722+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:13.724 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:13.824 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:13.823+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:13.875 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:13.876 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:13.884 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:13.890 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-18T01:03:13.925 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:13.924+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:13.947 INFO:teuthology.orchestra.run.smithi164.stdout:1823 read 0x2a17000 thru 0x2a1fe30 (0x8e31 bytes) 2024-09-18T01:03:13.948 INFO:teuthology.orchestra.run.smithi164.stdout:1824 write 0x306f400 thru 0x30706f2 (0x12f3 bytes) 2024-09-18T01:03:13.949 INFO:teuthology.orchestra.run.smithi164.stdout:1826 punch from 0x7ac6600 to 0x7acd400, (0x6e00 bytes) 2024-09-18T01:03:13.991 INFO:teuthology.orchestra.run.smithi164.stdout:1828 write 0x39b6200 thru 0x39bb14c (0x4f4d bytes) 2024-09-18T01:03:13.991 INFO:teuthology.orchestra.run.smithi164.stdout:1830 read 0xc2ce600 thru 0xc2dcf04 (0xe905 bytes) 2024-09-18T01:03:13.992 INFO:teuthology.orchestra.run.smithi164.stdout:1831 write 0x9760600 thru 0x9766423 (0x5e24 bytes) 2024-09-18T01:03:13.992 INFO:teuthology.orchestra.run.smithi164.stdout:1832 read 0x294e400 thru 0x2959059 (0xac5a bytes) 2024-09-18T01:03:13.994 INFO:teuthology.orchestra.run.smithi164.stdout:1837 write 0x6aaca00 thru 0x6aae504 (0x1b05 bytes) 2024-09-18T01:03:13.994 INFO:teuthology.orchestra.run.smithi164.stdout:1838 write 0xbdc7800 thru 0xbdc7db6 (0x5b7 bytes) 2024-09-18T01:03:13.994 INFO:teuthology.orchestra.run.smithi164.stdout:1840 punch from 0x558400 to 0x561800, (0x9400 bytes) 2024-09-18T01:03:14.025 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:14.024+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:14.041 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:14.042 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:14.054 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:14.062 INFO:teuthology.orchestra.run.smithi164.stdout:1841 punch from 0x16cd400 to 0x16d2000, (0x4c00 bytes) 2024-09-18T01:03:14.081 INFO:teuthology.orchestra.run.smithi164.stdout:1842 write 0xa5d0c00 thru 0xa5dda83 (0xce84 bytes) 2024-09-18T01:03:14.119 INFO:teuthology.orchestra.run.smithi164.stdout:1844 write 0xd10d600 thru 0xd10e64d (0x104e bytes) 2024-09-18T01:03:14.120 INFO:teuthology.orchestra.run.smithi164.stdout:1847 punch from 0x9125e00 to 0x912b400, (0x5600 bytes) 2024-09-18T01:03:14.127 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:14.126+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:14.137 INFO:teuthology.orchestra.run.smithi164.stdout:1848 trunc from 0xd10e800 to 0x1fdac00 2024-09-18T01:03:14.222 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:14.222 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:14.223 INFO:teuthology.orchestra.run.smithi164.stdout:1851 read 0x6bde00 thru 0x6c1b95 (0x3d96 bytes) 2024-09-18T01:03:14.226 INFO:teuthology.orchestra.run.smithi164.stdout:1852 trunc from 0x1fdac00 to 0x3a92c00 2024-09-18T01:03:14.228 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:14.227+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:14.231 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:14.249 INFO:teuthology.orchestra.run.smithi164.stdout:1853 trunc from 0x3a92c00 to 0xe461600 2024-09-18T01:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout:1854 read 0x7402a00 thru 0x7402c41 (0x242 bytes) 2024-09-18T01:03:14.260 INFO:teuthology.orchestra.run.smithi164.stdout:1855 punch from 0x77ec200 to 0x77ed200, (0x1000 bytes) 2024-09-18T01:03:14.272 INFO:teuthology.orchestra.run.smithi164.stdout:1856 trunc from 0xe461600 to 0x54d4400 2024-09-18T01:03:14.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:14.286 INFO:teuthology.orchestra.run.smithi164.stdout:1857 read 0x5235800 thru 0x5244e26 (0xf627 bytes) 2024-09-18T01:03:14.287 INFO:teuthology.orchestra.run.smithi164.stdout:1858 punch from 0x2210200 to 0x2213400, (0x3200 bytes) 2024-09-18T01:03:14.329 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:14.328+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:14.335 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:14.335 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:14.344 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:14.402 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:14.403 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:14.419 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:14.420 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:14.428 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:03:14.430 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:14.429+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:14.438 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:14.439 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:14.455 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:14.456 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:14.473 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:14.474 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:14.488 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:14.489 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:14.490 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 42.114376184999998, 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.116364879999999, 2024-09-18T01:03:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 40.108687928000002, 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.010829978, 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.154748966, 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.154379670000001, 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.152640824999999, 2024-09-18T01:03:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.152568935000001, 2024-09-18T01:03:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.064702924999999, 2024-09-18T01:03:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.058511666000001, 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.056981981, 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.056602292000001, 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.041103888999999, 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.040897130000001, 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.040479373, 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.040270135, 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.040206201, 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.039457995999999, 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.611 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:14.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:14.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.615 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.780519191, 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.616 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:14.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:14.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 26.779655572999999, 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:14.622 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:14.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:14.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:14.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:14.628 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:14.631 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:14.628+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:14.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout:1859 write 0x7184200 thru 0x7192d5f (0xeb60 bytes) 2024-09-18T01:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout:1860 read 0xdf1000 thru 0xe00fc2 (0xffc3 bytes) 2024-09-18T01:03:14.632 INFO:teuthology.orchestra.run.smithi164.stdout:1863 read 0xf94600 thru 0xf979f4 (0x33f5 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1866 punch from 0x35b3000 to 0x35b6a00, (0x3a00 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1868 write 0x6756800 thru 0x67620eb (0xb8ec bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1869 read 0x2bf5400 thru 0x2bfe0a8 (0x8ca9 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1870 read 0x310c200 thru 0x3118d37 (0xcb38 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1871 write 0x7797000 thru 0x779c5c2 (0x55c3 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1872 trunc from 0x779c600 to 0x518d800 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1873 write 0x2d2f800 thru 0x2d37ce8 (0x84e9 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1874 write 0x2de2600 thru 0x2defc34 (0xd635 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1881 punch from 0x459e00 to 0x469600, (0xf800 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1882 punch from 0x39e8a00 to 0x39f8400, (0xfa00 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1884 read 0x32f6200 thru 0x33014e4 (0xb2e5 bytes) 2024-09-18T01:03:14.633 INFO:teuthology.orchestra.run.smithi164.stdout:1885 punch from 0x3fc8c00 to 0x3fc9600, (0xa00 bytes) 2024-09-18T01:03:14.634 INFO:teuthology.orchestra.run.smithi164.stdout:1886 trunc from 0x518d800 to 0xb45c400 2024-09-18T01:03:14.674 INFO:teuthology.orchestra.run.smithi164.stdout:1887 punch from 0x474c400 to 0x4759600, (0xd200 bytes) 2024-09-18T01:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout:1889 read 0x477d800 thru 0x47829d6 (0x51d7 bytes) 2024-09-18T01:03:14.677 INFO:teuthology.orchestra.run.smithi164.stdout:1890 punch from 0x1aad800 to 0x1ab1000, (0x3800 bytes) 2024-09-18T01:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout:1893 punch from 0x7e80a00 to 0x7e8e000, (0xd600 bytes) 2024-09-18T01:03:14.680 INFO:teuthology.orchestra.run.smithi164.stdout:1894 read 0x14db600 thru 0x14e73ca (0xbdcb bytes) 2024-09-18T01:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout:1895 write 0x424f200 thru 0x4258fad (0x9dae bytes) 2024-09-18T01:03:14.689 INFO:teuthology.orchestra.run.smithi164.stdout:1896 read 0x52b4a00 thru 0x52b8335 (0x3936 bytes) 2024-09-18T01:03:14.693 INFO:teuthology.orchestra.run.smithi164.stdout:1897 write 0x16e8200 thru 0x16f660e (0xe40f bytes) 2024-09-18T01:03:14.694 INFO:teuthology.orchestra.run.smithi164.stdout:1900 trunc from 0xb45c400 to 0x9fdd600 2024-09-18T01:03:14.730 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:14.729+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:14.750 INFO:teuthology.orchestra.run.smithi164.stdout:1901 trunc from 0x9fdd600 to 0x970b200 2024-09-18T01:03:14.769 INFO:teuthology.orchestra.run.smithi164.stdout:1903 read 0x6f32c00 thru 0x6f3e2f7 (0xb6f8 bytes) 2024-09-18T01:03:14.769 INFO:teuthology.orchestra.run.smithi164.stdout:1904 punch from 0x6c8a400 to 0x6c95600, (0xb200 bytes) 2024-09-18T01:03:14.770 INFO:teuthology.orchestra.run.smithi164.stdout:1905 read 0x1390a00 thru 0x139cadb (0xc0dc bytes) 2024-09-18T01:03:14.771 INFO:teuthology.orchestra.run.smithi164.stdout:1906 read 0x1e86000 thru 0x1e86b95 (0xb96 bytes) 2024-09-18T01:03:14.772 INFO:teuthology.orchestra.run.smithi164.stdout:1908 read 0x67ac00 thru 0x689abb (0xeebc bytes) 2024-09-18T01:03:14.780 INFO:teuthology.orchestra.run.smithi164.stdout:1909 trunc from 0x970b200 to 0x5913200 2024-09-18T01:03:14.802 INFO:teuthology.orchestra.run.smithi164.stdout:1910 trunc from 0x5913200 to 0x6b32e00 2024-09-18T01:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4873.0:7609 3.17 3:e8c9d636:::rbd_object_map.11035907e8d0:head [call lock.assert_locked in=44b,call rbd.object_map_resize in=29b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-18T01:03:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:14.811415+0000", 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00034035899999999998, 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036674800000000002, 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for sub ops", 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4873", 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1716866284", 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7609 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811415+0000", 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811415+0000", 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811416+0000", 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.376e-06 2024-09-18T01:03:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811423+0000", 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7569999999999998e-06 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811437+0000", 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4018e-05 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811446+0000", 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811501+0000", 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5074999999999997e-05 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811557+0000", 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6657999999999999e-05 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:14.811734+0000", 2024-09-18T01:03:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176665 2024-09-18T01:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-18T01:03:14.817 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:14.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:14.824 INFO:teuthology.orchestra.run.smithi164.stdout:1911 trunc from 0x6b32e00 to 0x28a3400 2024-09-18T01:03:14.867 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:14.866+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:14.875 INFO:teuthology.orchestra.run.smithi164.stdout:1912 write 0xafc3a00 thru 0xafc9b10 (0x6111 bytes) 2024-09-18T01:03:14.875 INFO:teuthology.orchestra.run.smithi164.stdout:1913 read 0x72fe800 thru 0x7306c60 (0x8461 bytes) 2024-09-18T01:03:14.887 INFO:teuthology.orchestra.run.smithi164.stdout:1914 trunc from 0xafc9c00 to 0x13f6600 2024-09-18T01:03:14.931 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:14.930+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:14.939 INFO:teuthology.orchestra.run.smithi164.stdout:1917 read 0x8d4000 thru 0x8e0a18 (0xca19 bytes) 2024-09-18T01:03:14.963 INFO:teuthology.orchestra.run.smithi164.stdout:1919 write 0x37a1000 thru 0x37a69ec (0x59ed bytes) 2024-09-18T01:03:14.963 INFO:teuthology.orchestra.run.smithi164.stdout:1921 punch from 0x2539000 to 0x2545200, (0xc200 bytes) 2024-09-18T01:03:14.965 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:14.965 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:14.966 INFO:teuthology.orchestra.run.smithi164.stdout:1925 punch from 0x18c8000 to 0x18cb800, (0x3800 bytes) 2024-09-18T01:03:14.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:14.989 INFO:teuthology.orchestra.run.smithi164.stdout:1926 write 0x52da000 thru 0x52ded25 (0x4d26 bytes) 2024-09-18T01:03:14.994 INFO:teuthology.orchestra.run.smithi164.stdout:1928 trunc from 0x52dee00 to 0x229c400 2024-09-18T01:03:15.017 INFO:teuthology.orchestra.run.smithi164.stdout:1929 read 0xd44c00 thru 0xd4a070 (0x5471 bytes) 2024-09-18T01:03:15.022 INFO:teuthology.orchestra.run.smithi164.stdout:1930 punch from 0xd93600 to 0xd9cc00, (0x9600 bytes) 2024-09-18T01:03:15.024 INFO:teuthology.orchestra.run.smithi164.stdout:1936 punch from 0x2200600 to 0x2208a00, (0x8400 bytes) 2024-09-18T01:03:15.024 INFO:teuthology.orchestra.run.smithi164.stdout:1937 read 0x10c8400 thru 0x10d2870 (0xa471 bytes) 2024-09-18T01:03:15.033 INFO:teuthology.orchestra.run.smithi164.stdout:1938 trunc from 0x229c400 to 0x1739400 2024-09-18T01:03:15.062 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:15.061+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:15.069 INFO:teuthology.orchestra.run.smithi164.stdout:1939 read 0x175600 thru 0x17e640 (0x9041 bytes) 2024-09-18T01:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout:1940 punch from 0x1076400 to 0x1078c00, (0x2800 bytes) 2024-09-18T01:03:15.070 INFO:teuthology.orchestra.run.smithi164.stdout:1941 write 0x1080800 thru 0x10824dc (0x1cdd bytes) 2024-09-18T01:03:15.074 INFO:teuthology.orchestra.run.smithi164.stdout:1943 trunc from 0x1739400 to 0x3d21800 2024-09-18T01:03:15.085 INFO:teuthology.orchestra.run.smithi164.stdout:1944 read 0x3a64800 thru 0x3a6eb9c (0xa39d bytes) 2024-09-18T01:03:15.087 INFO:teuthology.orchestra.run.smithi164.stdout:1945 trunc from 0x3d21800 to 0x52fe000 2024-09-18T01:03:15.099 INFO:teuthology.orchestra.run.smithi164.stdout:1947 punch from 0xa46c00 to 0xa4aa00, (0x3e00 bytes) 2024-09-18T01:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout:1949 write 0x3b8b200 thru 0x3b90076 (0x4e77 bytes) 2024-09-18T01:03:15.101 INFO:teuthology.orchestra.run.smithi164.stdout:1950 punch from 0x394c800 to 0x395a800, (0xe000 bytes) 2024-09-18T01:03:15.126 INFO:teuthology.orchestra.run.smithi164.stdout:1951 write 0xc552200 thru 0xc559ec3 (0x7cc4 bytes) 2024-09-18T01:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout:1953 read 0x5283e00 thru 0x5285b6d (0x1d6e bytes) 2024-09-18T01:03:15.127 INFO:teuthology.orchestra.run.smithi164.stdout:1955 read 0x90b4200 thru 0x90bc9bd (0x87be bytes) 2024-09-18T01:03:15.132 INFO:teuthology.orchestra.run.smithi164.stdout:1956 trunc from 0xc55a000 to 0x82d6400 2024-09-18T01:03:15.133 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:15.132+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:15.155 INFO:teuthology.orchestra.run.smithi164.stdout:1959 read 0x30df400 thru 0x30eaf07 (0xbb08 bytes) 2024-09-18T01:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout:1960 write 0x3a71400 thru 0x3a74808 (0x3409 bytes) 2024-09-18T01:03:15.156 INFO:teuthology.orchestra.run.smithi164.stdout:1961 write 0x4718800 thru 0x471897e (0x17f bytes) 2024-09-18T01:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout:1962 read 0x54c0800 thru 0x54c8ad7 (0x82d8 bytes) 2024-09-18T01:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout:1965 write 0x574c00 thru 0x576beb (0x1fec bytes) 2024-09-18T01:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout:1966 write 0x7c5e400 thru 0x7c61c32 (0x3833 bytes) 2024-09-18T01:03:15.157 INFO:teuthology.orchestra.run.smithi164.stdout:1969 read 0x6c4a400 thru 0x6c4bf07 (0x1b08 bytes) 2024-09-18T01:03:15.160 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2015 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4265984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.840133201, 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027955569999999999, 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2015 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318446+0000", 2024-09-18T01:03:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-18T01:03:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318453+0000", 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318458+0000", 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9910000000000002e-06 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346240+0000", 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027782305 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346316+0000", 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5731999999999999e-05 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346399+0000", 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3137e-05 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.840112301000001, 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.840091791999999, 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.840071987000002, 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.840037346999999, 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.840015846, 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.839992669000001, 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2237 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4202496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.836023883999999, 2024-09-18T01:03:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028056793, 2024-09-18T01:03:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2237 2024-09-18T01:03:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322555+0000", 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7740000000000001e-06 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322562+0000", 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.426e-06 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322567+0000", 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350490+0000", 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027923436999999999 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350528+0000", 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7660999999999997e-05 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350610+0000", 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2150000000000005e-05 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:15.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.836003837, 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835984283999998, 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:15.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.83596378, 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:15.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:15.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835942916, 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:15.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:15.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835923015999999, 2024-09-18T01:03:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:15.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:15.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835898927999999, 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:15.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:15.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835867608000001, 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:15.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835823002000001, 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:15.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:15.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:15.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835792211000001, 2024-09-18T01:03:15.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:15.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.261 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835768726000001, 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835739670999999, 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.835718912000001, 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.279 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:15.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:15.282 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:15.279+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1972 trunc from 0x82d6400 to 0xc475600 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1974 punch from 0x99dd800 to 0x99ddc00, (0x400 bytes) 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1977 read 0xaeeb400 thru 0xaef5ac9 (0xa6ca bytes) 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1978 write 0xc1f5800 thru 0xc1fd59d (0x7d9e bytes) 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1981 write 0x1593400 thru 0x1598126 (0x4d27 bytes) 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1984 write 0xbd9600 thru 0xbde380 (0x4d81 bytes) 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1986 write 0x3644200 thru 0x3646c9d (0x2a9e bytes) 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1988 read 0x167e400 thru 0x1682307 (0x3f08 bytes) 2024-09-18T01:03:15.284 INFO:teuthology.orchestra.run.smithi164.stdout:1989 punch from 0xa502200 to 0xa503000, (0xe00 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:1991 punch from 0x77d3000 to 0x77d4800, (0x1800 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:1992 punch from 0x2508e00 to 0x2512a00, (0x9c00 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:1994 read 0x42dd000 thru 0x42eaf4c (0xdf4d bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:1995 write 0xdfc1200 thru 0xdfcaf50 (0x9d51 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:1996 trunc from 0xdfcb000 to 0xcfcec00 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:1998 punch from 0x1ef2c00 to 0x1effa00, (0xce00 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:1999 punch from 0xb7d5800 to 0xb7dd400, (0x7c00 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:2000 write 0x6f0e400 thru 0x6f1db88 (0xf789 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:2005 read 0x66d1800 thru 0x66d3042 (0x1843 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:2007 punch from 0x11b6800 to 0x11bb000, (0x4800 bytes) 2024-09-18T01:03:15.285 INFO:teuthology.orchestra.run.smithi164.stdout:2008 read 0x9c54600 thru 0x9c59b83 (0x5584 bytes) 2024-09-18T01:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout:2009 write 0xc511a00 thru 0xc51c076 (0xa677 bytes) 2024-09-18T01:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout:2010 read 0x95a0200 thru 0x95a6e83 (0x6c84 bytes) 2024-09-18T01:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout:2012 punch from 0x7625600 to 0x7628200, (0x2c00 bytes) 2024-09-18T01:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout:2013 read 0x7de0c00 thru 0x7de5be5 (0x4fe6 bytes) 2024-09-18T01:03:15.286 INFO:teuthology.orchestra.run.smithi164.stdout:2014 clone 13 order 18 su 16384 sc 15 2024-09-18T01:03:15.381 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:15.380+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:15.429 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:15.430 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:15.430 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:15.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:15.486 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:15.485+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:15.540 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone12 from 0xcfcec00 (overlap 0x13f6600) to 0x11f7400 2024-09-18T01:03:15.563 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:15.563 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:15.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:15.582 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:15.581+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:15.683 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:15.682+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:15.727 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.448645055, 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.444019151999999, 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.443941103, 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1292 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1774592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.387169377999999, 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050913359999999998, 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1292 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338601+0000", 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.024e-06 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338610+0000", 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5469999999999997e-06 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338616+0000", 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389393+0000", 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05077661 2024-09-18T01:03:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389472+0000", 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9776000000000003e-05 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389512+0000", 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0000000000000003e-05 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.387145060000002, 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.387119196, 2024-09-18T01:03:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.387096715999999, 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.387075679999999, 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.387048285999999, 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.387023980999999, 2024-09-18T01:03:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383624498, 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383599938, 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383558421, 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383537918999998, 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.799 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383517453, 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383476113, 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:15.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383456075000002, 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:15.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:15.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383413743999999, 2024-09-18T01:03:15.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:15.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:15.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:15.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:15.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.821 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.821 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.821 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383392751999999, 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.383341781999999, 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:15.830 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:15.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:15.832 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:15.830+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:15.885 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:15.885 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:15.885 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T01:03:15.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:03:15.958 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-18T01:03:15.960 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:15.959+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:15.984 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:15.984 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:15.984 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:15.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:16.028 INFO:teuthology.orchestra.run.smithi164.stdout:2015 trunc from 0xcfcec00 to 0x3549000 2024-09-18T01:03:16.033 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:16.032+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:16.069 INFO:teuthology.orchestra.run.smithi164.stdout:2017 read 0x34d2a00 thru 0x34df056 (0xc657 bytes) 2024-09-18T01:03:16.084 INFO:teuthology.orchestra.run.smithi164.stdout:2019 write 0x4175200 thru 0x41782db (0x30dc bytes) 2024-09-18T01:03:16.084 INFO:teuthology.orchestra.run.smithi164.stdout:2021 read 0xbcaa00 thru 0xbd9b93 (0xf194 bytes) 2024-09-18T01:03:16.107 INFO:teuthology.orchestra.run.smithi164.stdout:2022 write 0x4fe9e00 thru 0x4feed6d (0x4f6e bytes) 2024-09-18T01:03:16.107 INFO:teuthology.orchestra.run.smithi164.stdout:2023 read 0x11200 thru 0x182c6 (0x70c7 bytes) 2024-09-18T01:03:16.108 INFO:teuthology.orchestra.run.smithi164.stdout:2024 read 0x3b35c00 thru 0x3b39b64 (0x3f65 bytes) 2024-09-18T01:03:16.126 INFO:teuthology.orchestra.run.smithi164.stdout:2027 write 0x86c3000 thru 0x86c512a (0x212b bytes) 2024-09-18T01:03:16.133 INFO:teuthology.orchestra.run.smithi164.stdout:2030 write 0x5994400 thru 0x599b31a (0x6f1b bytes) 2024-09-18T01:03:16.133 INFO:teuthology.orchestra.run.smithi164.stdout:2031 write 0x8392000 thru 0x83a180e (0xf80f bytes) 2024-09-18T01:03:16.133 INFO:teuthology.orchestra.run.smithi164.stdout:2033 punch from 0x3dfb200 to 0x3dfb800, (0x600 bytes) 2024-09-18T01:03:16.133 INFO:teuthology.orchestra.run.smithi164.stdout:2034 punch from 0x703c000 to 0x7044e00, (0x8e00 bytes) 2024-09-18T01:03:16.134 INFO:teuthology.orchestra.run.smithi164.stdout:2037 punch from 0x53c200 to 0x544600, (0x8400 bytes) 2024-09-18T01:03:16.134 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:16.133+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:16.135 INFO:teuthology.orchestra.run.smithi164.stdout:2039 read 0x6445400 thru 0x64535f2 (0xe1f3 bytes) 2024-09-18T01:03:16.135 INFO:teuthology.orchestra.run.smithi164.stdout:2042 punch from 0x1355600 to 0x1364000, (0xea00 bytes) 2024-09-18T01:03:16.136 INFO:teuthology.orchestra.run.smithi164.stdout:2043 clone 14 order 25 su 4194304 sc 12 2024-09-18T01:03:16.175 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:16.176 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:16.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 48, 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-18T01:02:13.493841+0000", 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-18T01:03:15.424027+0000", 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-18T01:02:24.290620+0000", 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-18T01:02:48.274416+0000", 2024-09-18T01:03:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-18T01:03:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:24.388064+0000", 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-18T01:03:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-18T01:03:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-18T01:03:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:30.676199+0000", 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-18T01:03:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2699999809265137, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2699999809265137, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:40.390732+0000", 2024-09-18T01:03:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "48", 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 15, 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 48, 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.5099999904632568, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.5099999904632568, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.25, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.25, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4563fb16-8745-4e75-b654-1e96403d91b9", 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-18T01:03:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/1626166637", 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/1626166637", 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/1626166637", 2024-09-18T01:03:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/1626166637", 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "634965cb-d444-4721-b34a-afbc49105cab", 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/2210970259", 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/2210970259", 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/2210970259", 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/2210970259", 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "83443eab-2fa4-4c7e-a459-edb1bb0ed8d0", 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/617133675", 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/617133675", 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/617133675", 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/617133675", 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "5797982a-71c1-4d6a-9604-00fbc6bd16d9", 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6808", 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6809", 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6810", 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6811", 2024-09-18T01:03:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6814", 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6815", 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6812", 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6813", 2024-09-18T01:03:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6809/151031464", 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6811/151031464", 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6815/151031464", 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6813/151031464", 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f79e3329-5481-4f33-8d28-6386af186492", 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6816", 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6817", 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6818", 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6819", 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6822", 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6823", 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6820", 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6821", 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6817/1499036965", 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6819/1499036965", 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6823/1499036965", 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6821/1499036965", 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "8c61d2ec-b111-4783-a016-336ed33a0694", 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 20, 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6800", 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6801", 2024-09-18T01:03:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6802", 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6803", 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6806", 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6807", 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6804", 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6805", 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6801/2310498524", 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6803/2310498524", 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6807/2310498524", 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6805/2310498524", 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.877700+0000", 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.360930+0000", 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.403205+0000", 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.952123+0000", 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:22.099545+0000", 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.694132+0000", 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-18T01:03:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-18T01:03:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.315 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:16.316 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:16.249+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:16.317 INFO:tasks.thrashosds.thrasher:j is {'epoch': 48, 'fsid': '62b46b83-8ef2-466f-b28b-2d97f265422e', 'created': '2024-09-18T01:02:13.493841+0000', 'modified': '2024-09-18T01:03:15.424027+0000', 'last_up_change': '2024-09-18T01:02:24.290620+0000', 'last_in_change': '2024-09-18T01:02:48.274416+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:02:24.388064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:02:30.676199+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:02:40.390732+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 48, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4563fb16-8745-4e75-b654-1e96403d91b9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 1626166637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 1626166637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 1626166637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 1626166637}]}, 'public_addr': '172.21.15.31:6817/1626166637', 'cluster_addr': '172.21.15.31:6819/1626166637', 'heartbeat_back_addr': '172.21.15.31:6823/1626166637', 'heartbeat_front_addr': '172.21.15.31:6821/1626166637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '634965cb-d444-4721-b34a-afbc49105cab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 2210970259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 2210970259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 2210970259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 2210970259}]}, 'public_addr': '172.21.15.31:6801/2210970259', 'cluster_addr': '172.21.15.31:6803/2210970259', 'heartbeat_back_addr': '172.21.15.31:6807/2210970259', 'heartbeat_front_addr': '172.21.15.31:6805/2210970259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 617133675}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 617133675}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 617133675}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 617133675}]}, 'public_addr': '172.21.15.31:6809/617133675', 'cluster_addr': '172.21.15.31:6811/617133675', 'heartbeat_back_addr': '172.21.15.31:6815/617133675', 'heartbeat_front_addr': '172.21.15.31:6813/617133675', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6808', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6809', 'nonce': 151031464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6810', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6811', 'nonce': 151031464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6814', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6815', 'nonce': 151031464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6812', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6813', 'nonce': 151031464}]}, 'public_addr': '172.21.15.98:6809/151031464', 'cluster_addr': '172.21.15.98:6811/151031464', 'heartbeat_back_addr': '172.21.15.98:6815/151031464', 'heartbeat_front_addr': '172.21.15.98:6813/151031464', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f79e3329-5481-4f33-8d28-6386af186492', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6816', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6817', 'nonce': 1499036965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6818', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6819', 'nonce': 1499036965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6822', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6823', 'nonce': 1499036965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6820', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6821', 'nonce': 1499036965}]}, 'public_addr': '172.21.15.98:6817/1499036965', 'cluster_addr': '172.21.15.98:6819/1499036965', 'heartbeat_back_addr': '172.21.15.98:6823/1499036965', 'heartbeat_front_addr': '172.21.15.98:6821/1499036965', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8c61d2ec-b111-4783-a016-336ed33a0694', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6800', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6801', 'nonce': 2310498524}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6802', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6803', 'nonce': 2310498524}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6806', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6807', 'nonce': 2310498524}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6804', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6805', 'nonce': 2310498524}]}, 'public_addr': '172.21.15.98:6801/2310498524', 'cluster_addr': '172.21.15.98:6803/2310498524', 'heartbeat_back_addr': '172.21.15.98:6807/2310498524', 'heartbeat_front_addr': '172.21.15.98:6805/2310498524', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.877700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.360930+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.403205+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.952123+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:22.099545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.694132+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:03:16.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:03:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.597806215, 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.597786256999999, 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.597767316000001, 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.597745807000001, 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.597720079, 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588988219999999, 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588965505999999, 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588946375999999, 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588927529999999, 2024-09-18T01:03:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588908652000001, 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588886787, 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588863152, 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588829501999999, 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588801647, 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588781824, 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.58876139, 2024-09-18T01:03:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588739309999999, 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588716988, 2024-09-18T01:03:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588694592, 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588674172999999, 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:16.463 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:16.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:03:16.465 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:16.466 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:16.464+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:16.562 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone13 intact 2024-09-18T01:03:16.565 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:16.564+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:16.574 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:16.574 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:16.574 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:16.574 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:16.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:16.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:16.667 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:16.666+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:16.728 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:16.728 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:16.728 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:16.728 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:16.728 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:16.737 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:16.762 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:16.762 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:16.774 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:16.775 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:03:16.778 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:16.777+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:16.780 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-18T01:03:16.434813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490167465,"num_objects":538,"num_object_clones":76,"num_object_copies":1076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":538,"num_whiteouts":239,"num_read":104260,"num_read_kb":347682,"num_write":9013,"num_write_kb":95991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":81996,"num_keys_recovered":14,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7512,"ondisk_log_size":7512,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":224848,"kb_used_data":17592,"kb_used_omap":7,"kb_used_meta":207160,"kb_avail":471634352,"statfs":{"total":483183820800,"available":482953576448,"internally_reserved":0,"allocated":18014208,"data_stored":121477141,"data_compressed":2063872,"data_compressed_allocated":9785344,"data_compressed_original":116413440,"omap_allocated":7973,"internal_metadata":212132059},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":211003162,"num_objects":119,"num_object_clones":53,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":59,"num_read":19213,"num_read_kb":188482,"num_write":859,"num_write_kb":45580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.084558"},"pg_stats":[{"pgid":"3.1f","version":"46'119","reported_seq":1883,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.229964+0000","last_change":"2024-09-18T01:02:43.469611+0000","last_active":"2024-09-18T01:03:14.229964+0000","last_peered":"2024-09-18T01:03:14.229964+0000","last_clean":"2024-09-18T01:03:14.229964+0000","last_became_active":"2024-09-18T01:02:41.451184+0000","last_became_peered":"2024-09-18T01:02:41.451184+0000","last_unstale":"2024-09-18T01:03:14.229964+0000","last_undegraded":"2024-09-18T01:03:14.229964+0000","last_fullsized":"2024-09-18T01:03:14.229964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:58:03.781484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005167,"stat_sum":{"num_bytes":31524880,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1724,"num_read_kb":23659,"num_write":148,"num_write_kb":927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"46'136","reported_seq":687,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.126163+0000","last_change":"2024-09-18T01:02:43.469963+0000","last_active":"2024-09-18T01:03:14.126163+0000","last_peered":"2024-09-18T01:03:14.126163+0000","last_clean":"2024-09-18T01:03:14.126163+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:03:14.126163+0000","last_undegraded":"2024-09-18T01:03:14.126163+0000","last_fullsized":"2024-09-18T01:03:14.126163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:49:58.302440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013026730000000001,"stat_sum":{"num_bytes":1334272,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":498,"num_read_kb":620,"num_write":132,"num_write_kb":926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"46'177","reported_seq":4510,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.293052+0000","last_change":"2024-09-18T01:02:49.322327+0000","last_active":"2024-09-18T01:03:14.293052+0000","last_peered":"2024-09-18T01:03:14.293052+0000","last_clean":"2024-09-18T01:03:14.293052+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:03:14.293052+0000","last_undegraded":"2024-09-18T01:03:14.293052+0000","last_fullsized":"2024-09-18T01:03:14.293052+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:48:55.058786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918460000000001,"stat_sum":{"num_bytes":11611323,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":4868,"num_read_kb":12899,"num_write":173,"num_write_kb":5029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"46'223","reported_seq":4020,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.272604+0000","last_change":"2024-09-18T01:02:49.301865+0000","last_active":"2024-09-18T01:03:14.272604+0000","last_peered":"2024-09-18T01:03:14.272604+0000","last_clean":"2024-09-18T01:03:14.272604+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:03:14.272604+0000","last_undegraded":"2024-09-18T01:03:14.272604+0000","last_fullsized":"2024-09-18T01:03:14.272604+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.541307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242879999999999,"stat_sum":{"num_bytes":24559145,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":4813,"num_read_kb":12252,"num_write":177,"num_write_kb":4641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"46'267","reported_seq":852,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.148269+0000","last_change":"2024-09-18T01:02:43.469715+0000","last_active":"2024-09-18T01:03:14.148269+0000","last_peered":"2024-09-18T01:03:14.148269+0000","last_clean":"2024-09-18T01:03:14.148269+0000","last_became_active":"2024-09-18T01:02:41.454655+0000","last_became_peered":"2024-09-18T01:02:41.454655+0000","last_unstale":"2024-09-18T01:03:14.148269+0000","last_undegraded":"2024-09-18T01:03:14.148269+0000","last_fullsized":"2024-09-18T01:03:14.148269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:01.654282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087094899999999999,"stat_sum":{"num_bytes":8915041,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":1192,"num_read_kb":1176,"num_write":398,"num_write_kb":1525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"46'132","reported_seq":1617,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.286624+0000","last_change":"2024-09-18T01:02:43.474216+0000","last_active":"2024-09-18T01:03:14.286624+0000","last_peered":"2024-09-18T01:03:14.286624+0000","last_clean":"2024-09-18T01:03:14.286624+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:03:14.286624+0000","last_undegraded":"2024-09-18T01:03:14.286624+0000","last_fullsized":"2024-09-18T01:03:14.286624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:08.372810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074574000000000005,"stat_sum":{"num_bytes":3674695,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":1442,"num_read_kb":2439,"num_write":126,"num_write_kb":4121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"46'373","reported_seq":4311,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.124506+0000","last_change":"2024-09-18T01:02:43.475997+0000","last_active":"2024-09-18T01:03:14.124506+0000","last_peered":"2024-09-18T01:03:14.124506+0000","last_clean":"2024-09-18T01:03:14.124506+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:03:14.124506+0000","last_undegraded":"2024-09-18T01:03:14.124506+0000","last_fullsized":"2024-09-18T01:03:14.124506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:45:47.443561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038270300000000003,"stat_sum":{"num_bytes":11478057,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":6073,"num_read_kb":11471,"num_write":370,"num_write_kb":3583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"46'168","reported_seq":2324,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.101458+0000","last_change":"2024-09-18T01:02:43.468721+0000","last_active":"2024-09-18T01:03:14.101458+0000","last_peered":"2024-09-18T01:03:14.101458+0000","last_clean":"2024-09-18T01:03:14.101458+0000","last_became_active":"2024-09-18T01:02:41.459321+0000","last_became_peered":"2024-09-18T01:02:41.459321+0000","last_unstale":"2024-09-18T01:03:14.101458+0000","last_undegraded":"2024-09-18T01:03:14.101458+0000","last_fullsized":"2024-09-18T01:03:14.101458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:23.508238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061623500000000002,"stat_sum":{"num_bytes":35992576,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":2119,"num_read_kb":32092,"num_write":182,"num_write_kb":9474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'225","reported_seq":3039,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.297371+0000","last_change":"2024-09-18T01:02:49.315232+0000","last_active":"2024-09-18T01:03:14.297371+0000","last_peered":"2024-09-18T01:03:14.297371+0000","last_clean":"2024-09-18T01:03:14.297371+0000","last_became_active":"2024-09-18T01:02:49.293095+0000","last_became_peered":"2024-09-18T01:02:49.293095+0000","last_unstale":"2024-09-18T01:03:14.297371+0000","last_undegraded":"2024-09-18T01:03:14.297371+0000","last_fullsized":"2024-09-18T01:03:14.297371+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:46.084218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097728599999999995,"stat_sum":{"num_bytes":24425574,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":3112,"num_read_kb":10372,"num_write":268,"num_write_kb":1242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"44'124","reported_seq":1515,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.422683+0000","last_change":"2024-09-18T01:02:49.351291+0000","last_active":"2024-09-18T01:03:13.422683+0000","last_peered":"2024-09-18T01:03:13.422683+0000","last_clean":"2024-09-18T01:03:13.422683+0000","last_became_active":"2024-09-18T01:02:49.291045+0000","last_became_peered":"2024-09-18T01:02:49.291045+0000","last_unstale":"2024-09-18T01:03:13.422683+0000","last_undegraded":"2024-09-18T01:03:13.422683+0000","last_fullsized":"2024-09-18T01:03:13.422683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:12.786190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099035500000000005,"stat_sum":{"num_bytes":8365665,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1325,"num_read_kb":10392,"num_write":122,"num_write_kb":4625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"46'142","reported_seq":4211,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.294098+0000","last_change":"2024-09-18T01:02:43.464371+0000","last_active":"2024-09-18T01:03:14.294098+0000","last_peered":"2024-09-18T01:03:14.294098+0000","last_clean":"2024-09-18T01:03:14.294098+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:03:14.294098+0000","last_undegraded":"2024-09-18T01:03:14.294098+0000","last_fullsized":"2024-09-18T01:03:14.294098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:02.730348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077757900000000003,"stat_sum":{"num_bytes":16999557,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":4053,"num_read_kb":19937,"num_write":131,"num_write_kb":736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"46'304","reported_seq":2720,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.229208+0000","last_change":"2024-09-18T01:02:43.464338+0000","last_active":"2024-09-18T01:03:14.229208+0000","last_peered":"2024-09-18T01:03:14.229208+0000","last_clean":"2024-09-18T01:03:14.229208+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:03:14.229208+0000","last_undegraded":"2024-09-18T01:03:14.229208+0000","last_fullsized":"2024-09-18T01:03:14.229208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:16.283116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080305299999999997,"stat_sum":{"num_bytes":8594524,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":3672,"num_read_kb":3374,"num_write":373,"num_write_kb":822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.422751+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:03:13.422751+0000","last_peered":"2024-09-18T01:03:13.422751+0000","last_clean":"2024-09-18T01:03:13.422751+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:03:13.422751+0000","last_undegraded":"2024-09-18T01:03:13.422751+0000","last_fullsized":"2024-09-18T01:03:13.422751+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"46'117","reported_seq":4202,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.090480+0000","last_change":"2024-09-18T01:02:43.469252+0000","last_active":"2024-09-18T01:03:14.090480+0000","last_peered":"2024-09-18T01:03:14.090480+0000","last_clean":"2024-09-18T01:03:14.090480+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:03:14.090480+0000","last_undegraded":"2024-09-18T01:03:14.090480+0000","last_fullsized":"2024-09-18T01:03:14.090480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:18.196436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066086499999999996,"stat_sum":{"num_bytes":64887808,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":4039,"num_read_kb":46256,"num_write":116,"num_write_kb":8462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.490689+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:03:13.490689+0000","last_peered":"2024-09-18T01:03:13.490689+0000","last_clean":"2024-09-18T01:03:13.490689+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:03:13.490689+0000","last_undegraded":"2024-09-18T01:03:13.490689+0000","last_fullsized":"2024-09-18T01:03:13.490689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"46'1411","reported_seq":4582,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.301046+0000","last_change":"2024-09-18T01:02:43.469995+0000","last_active":"2024-09-18T01:03:14.301046+0000","last_peered":"2024-09-18T01:03:14.301046+0000","last_clean":"2024-09-18T01:03:14.301046+0000","last_became_active":"2024-09-18T01:02:41.456801+0000","last_became_peered":"2024-09-18T01:02:41.456801+0000","last_unstale":"2024-09-18T01:03:14.301046+0000","last_undegraded":"2024-09-18T01:03:14.301046+0000","last_fullsized":"2024-09-18T01:03:14.301046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:54:21.617325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159869,"stat_sum":{"num_bytes":14486754,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":10559,"num_read_kb":20914,"num_write":2579,"num_write_kb":3036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.422673+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:03:13.422673+0000","last_peered":"2024-09-18T01:03:13.422673+0000","last_clean":"2024-09-18T01:03:13.422673+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:03:13.422673+0000","last_undegraded":"2024-09-18T01:03:13.422673+0000","last_fullsized":"2024-09-18T01:03:13.422673+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'157","reported_seq":3430,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.490352+0000","last_change":"2024-09-18T01:02:43.476334+0000","last_active":"2024-09-18T01:03:13.490352+0000","last_peered":"2024-09-18T01:03:13.490352+0000","last_clean":"2024-09-18T01:03:13.490352+0000","last_became_active":"2024-09-18T01:02:41.457022+0000","last_became_peered":"2024-09-18T01:02:41.457022+0000","last_unstale":"2024-09-18T01:03:13.490352+0000","last_undegraded":"2024-09-18T01:03:13.490352+0000","last_fullsized":"2024-09-18T01:03:13.490352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:53.506455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050705100000000001,"stat_sum":{"num_bytes":9590784,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":3585,"num_read_kb":6603,"num_write":146,"num_write_kb":945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.490550+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:03:13.490550+0000","last_peered":"2024-09-18T01:03:13.490550+0000","last_clean":"2024-09-18T01:03:13.490550+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:03:13.490550+0000","last_undegraded":"2024-09-18T01:03:13.490550+0000","last_fullsized":"2024-09-18T01:03:13.490550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"46'119","reported_seq":4013,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.295399+0000","last_change":"2024-09-18T01:02:49.294263+0000","last_active":"2024-09-18T01:03:14.295399+0000","last_peered":"2024-09-18T01:03:14.295399+0000","last_clean":"2024-09-18T01:03:14.295399+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:03:14.295399+0000","last_undegraded":"2024-09-18T01:03:14.295399+0000","last_fullsized":"2024-09-18T01:03:14.295399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:49:02.651919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13052448,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":3875,"num_read_kb":14169,"num_write":141,"num_write_kb":7353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"46'235","reported_seq":3674,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.269960+0000","last_change":"2024-09-18T01:02:43.473795+0000","last_active":"2024-09-18T01:03:14.269960+0000","last_peered":"2024-09-18T01:03:14.269960+0000","last_clean":"2024-09-18T01:03:14.269960+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:03:14.269960+0000","last_undegraded":"2024-09-18T01:03:14.269960+0000","last_fullsized":"2024-09-18T01:03:14.269960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:30.087109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031347599999999998,"stat_sum":{"num_bytes":30527488,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":4596,"num_read_kb":27260,"num_write":181,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":67,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.490317+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:03:13.490317+0000","last_peered":"2024-09-18T01:03:13.490317+0000","last_clean":"2024-09-18T01:03:13.490317+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:03:13.490317+0000","last_undegraded":"2024-09-18T01:03:13.490317+0000","last_fullsized":"2024-09-18T01:03:13.490317+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"46'88","reported_seq":2405,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.295027+0000","last_change":"2024-09-18T01:02:43.476102+0000","last_active":"2024-09-18T01:03:14.295027+0000","last_peered":"2024-09-18T01:03:14.295027+0000","last_clean":"2024-09-18T01:03:14.295027+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:03:14.295027+0000","last_undegraded":"2024-09-18T01:03:14.295027+0000","last_fullsized":"2024-09-18T01:03:14.295027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:42.570019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043811300000000001,"stat_sum":{"num_bytes":30239490,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":2296,"num_read_kb":11641,"num_write":87,"num_write_kb":2405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.424188+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:03:13.424188+0000","last_peered":"2024-09-18T01:03:13.424188+0000","last_clean":"2024-09-18T01:03:13.424188+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:03:13.424188+0000","last_undegraded":"2024-09-18T01:03:13.424188+0000","last_fullsized":"2024-09-18T01:03:13.424188+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'194","reported_seq":1954,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.297084+0000","last_change":"2024-09-18T01:02:43.468762+0000","last_active":"2024-09-18T01:03:14.297084+0000","last_peered":"2024-09-18T01:03:14.297084+0000","last_clean":"2024-09-18T01:03:14.297084+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:03:14.297084+0000","last_undegraded":"2024-09-18T01:03:14.297084+0000","last_fullsized":"2024-09-18T01:03:14.297084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:17:25.263418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296199999999997,"stat_sum":{"num_bytes":15233024,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":2375,"num_read_kb":7873,"num_write":166,"num_write_kb":6125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.422310+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:03:13.422310+0000","last_peered":"2024-09-18T01:03:13.422310+0000","last_clean":"2024-09-18T01:03:13.422310+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:03:13.422310+0000","last_undegraded":"2024-09-18T01:03:13.422310+0000","last_fullsized":"2024-09-18T01:03:13.422310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"46'239","reported_seq":2379,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.264487+0000","last_change":"2024-09-18T01:02:49.337150+0000","last_active":"2024-09-18T01:03:14.264487+0000","last_peered":"2024-09-18T01:03:14.264487+0000","last_clean":"2024-09-18T01:03:14.264487+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:03:14.264487+0000","last_undegraded":"2024-09-18T01:03:14.264487+0000","last_fullsized":"2024-09-18T01:03:14.264487+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:01:34.321156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049608300000000003,"stat_sum":{"num_bytes":3651175,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":3204,"num_read_kb":3539,"num_write":200,"num_write_kb":487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":53,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.424138+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:03:13.424138+0000","last_peered":"2024-09-18T01:03:13.424138+0000","last_clean":"2024-09-18T01:03:13.424138+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:03:13.424138+0000","last_undegraded":"2024-09-18T01:03:13.424138+0000","last_fullsized":"2024-09-18T01:03:13.424138+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":227,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.422315+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:03:13.422315+0000","last_peered":"2024-09-18T01:03:13.422315+0000","last_clean":"2024-09-18T01:03:13.422315+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:03:13.422315+0000","last_undegraded":"2024-09-18T01:03:13.422315+0000","last_fullsized":"2024-09-18T01:03:13.422315+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"46'172","reported_seq":3849,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.125906+0000","last_change":"2024-09-18T01:02:43.468997+0000","last_active":"2024-09-18T01:03:14.125906+0000","last_peered":"2024-09-18T01:03:14.125906+0000","last_clean":"2024-09-18T01:03:14.125906+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:03:14.125906+0000","last_undegraded":"2024-09-18T01:03:14.125906+0000","last_fullsized":"2024-09-18T01:03:14.125906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:31.806180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063056400000000002,"stat_sum":{"num_bytes":16566873,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":3911,"num_read_kb":7927,"num_write":210,"num_write_kb":4161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"46'185","reported_seq":2064,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:13.557770+0000","last_change":"2024-09-18T01:02:43.474261+0000","last_active":"2024-09-18T01:03:13.557770+0000","last_peered":"2024-09-18T01:03:13.557770+0000","last_clean":"2024-09-18T01:03:13.557770+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:03:13.557770+0000","last_undegraded":"2024-09-18T01:03:13.557770+0000","last_fullsized":"2024-09-18T01:03:13.557770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:15.474663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065863900000000003,"stat_sum":{"num_bytes":1538078,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":2201,"num_read_kb":1269,"num_write":232,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"46'463","reported_seq":2186,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.295136+0000","last_change":"2024-09-18T01:02:43.464113+0000","last_active":"2024-09-18T01:03:14.295136+0000","last_peered":"2024-09-18T01:03:14.295136+0000","last_clean":"2024-09-18T01:03:14.295136+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:03:14.295136+0000","last_undegraded":"2024-09-18T01:03:14.295136+0000","last_fullsized":"2024-09-18T01:03:14.295136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:23:44.659148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066269099999999997,"stat_sum":{"num_bytes":4108372,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":3471,"num_read_kb":3658,"num_write":648,"num_write_kb":1993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"46'140","reported_seq":598,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.231577+0000","last_change":"2024-09-18T01:02:43.476485+0000","last_active":"2024-09-18T01:03:14.231577+0000","last_peered":"2024-09-18T01:03:14.231577+0000","last_clean":"2024-09-18T01:03:14.231577+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:03:14.231577+0000","last_undegraded":"2024-09-18T01:03:14.231577+0000","last_fullsized":"2024-09-18T01:03:14.231577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:21.930593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054801899999999996,"stat_sum":{"num_bytes":2379808,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":413,"num_read_kb":1336,"num_write":132,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"46'131","reported_seq":1769,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.271784+0000","last_change":"2024-09-18T01:02:49.302821+0000","last_active":"2024-09-18T01:03:14.271784+0000","last_peered":"2024-09-18T01:03:14.271784+0000","last_clean":"2024-09-18T01:03:14.271784+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:03:14.271784+0000","last_undegraded":"2024-09-18T01:03:14.271784+0000","last_fullsized":"2024-09-18T01:03:14.271784+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:48:11.538644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043103,"stat_sum":{"num_bytes":10445824,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":1567,"num_read_kb":7081,"num_write":125,"num_write_kb":1785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"46'192","reported_seq":3048,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.204021+0000","last_change":"2024-09-18T01:02:43.464136+0000","last_active":"2024-09-18T01:03:14.204021+0000","last_peered":"2024-09-18T01:03:14.204021+0000","last_clean":"2024-09-18T01:03:14.204021+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:03:14.204021+0000","last_undegraded":"2024-09-18T01:03:14.204021+0000","last_fullsized":"2024-09-18T01:03:14.204021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:38.560269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061418400000000004,"stat_sum":{"num_bytes":9805312,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":3333,"num_read_kb":10381,"num_write":171,"num_write_kb":549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"46'102","reported_seq":2613,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.295239+0000","last_change":"2024-09-18T01:02:49.312596+0000","last_active":"2024-09-18T01:03:14.295239+0000","last_peered":"2024-09-18T01:03:14.295239+0000","last_clean":"2024-09-18T01:03:14.295239+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:03:14.295239+0000","last_undegraded":"2024-09-18T01:03:14.295239+0000","last_fullsized":"2024-09-18T01:03:14.295239+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:58:40.456869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013236470000000001,"stat_sum":{"num_bytes":16817713,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":2463,"num_read_kb":2379,"num_write":95,"num_write_kb":1256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"46'341","reported_seq":10302,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.213340+0000","last_change":"2024-09-18T01:02:49.321182+0000","last_active":"2024-09-18T01:03:14.213340+0000","last_peered":"2024-09-18T01:03:14.213340+0000","last_clean":"2024-09-18T01:03:14.213340+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:03:14.213340+0000","last_undegraded":"2024-09-18T01:03:14.213340+0000","last_fullsized":"2024-09-18T01:03:14.213340+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:04:56.724688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011817170000000001,"stat_sum":{"num_bytes":4923919,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":8262,"num_read_kb":11274,"num_write":197,"num_write_kb":739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"46'130","reported_seq":637,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.208570+0000","last_change":"2024-09-18T01:02:49.345428+0000","last_active":"2024-09-18T01:03:14.208570+0000","last_peered":"2024-09-18T01:03:14.208570+0000","last_clean":"2024-09-18T01:03:14.208570+0000","last_became_active":"2024-09-18T01:02:49.296123+0000","last_became_peered":"2024-09-18T01:02:49.296123+0000","last_unstale":"2024-09-18T01:03:14.208570+0000","last_undegraded":"2024-09-18T01:03:14.208570+0000","last_fullsized":"2024-09-18T01:03:14.208570+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:37:14.098253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053670300000000003,"stat_sum":{"num_bytes":3650824,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":462,"num_read_kb":1688,"num_write":121,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"46'141","reported_seq":1733,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.209487+0000","last_change":"2024-09-18T01:02:43.463798+0000","last_active":"2024-09-18T01:03:14.209487+0000","last_peered":"2024-09-18T01:03:14.209487+0000","last_clean":"2024-09-18T01:03:14.209487+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:03:14.209487+0000","last_undegraded":"2024-09-18T01:03:14.209487+0000","last_fullsized":"2024-09-18T01:03:14.209487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:34.327485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042487699999999999,"stat_sum":{"num_bytes":884224,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":1969,"num_read_kb":1954,"num_write":125,"num_write_kb":282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"46'311","reported_seq":4227,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.295840+0000","last_change":"2024-09-18T01:02:43.468742+0000","last_active":"2024-09-18T01:03:14.295840+0000","last_peered":"2024-09-18T01:03:14.295840+0000","last_clean":"2024-09-18T01:03:14.295840+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:03:14.295840+0000","last_undegraded":"2024-09-18T01:03:14.295840+0000","last_fullsized":"2024-09-18T01:03:14.295840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:48:24.063772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043475499999999999,"stat_sum":{"num_bytes":33023067,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":4740,"num_read_kb":14661,"num_write":446,"num_write_kb":6787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"46'134","reported_seq":2111,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-18T01:03:14.291885+0000","last_change":"2024-09-18T01:02:49.356374+0000","last_active":"2024-09-18T01:03:14.291885+0000","last_peered":"2024-09-18T01:03:14.291885+0000","last_clean":"2024-09-18T01:03:14.291885+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:03:14.291885+0000","last_undegraded":"2024-09-18T01:03:14.291885+0000","last_fullsized":"2024-09-18T01:03:14.291885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:35.518072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311600000000002,"stat_sum":{"num_bytes":16288784,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":1931,"num_read_kb":5035,"num_write":134,"num_write_kb":4038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":489577078,"num_objects":534,"num_object_clones":76,"num_object_copies":1068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":534,"num_whiteouts":239,"num_read":104143,"num_read_kb":347581,"num_write":8852,"num_write_kb":93573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":81977,"num_keys_recovered":14,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15302656,"data_stored":126398700,"data_compressed":2148142,"data_compressed_allocated":9773056,"data_compressed_original":122426368,"omap_allocated":0,"internal_metadata":0},"log_size":7392,"ondisk_log_size":7392,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640268,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640270,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42552,"kb_used_data":3044,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329288,"statfs":{"total":96636764160,"available":96593190912,"internally_reserved":0,"allocated":3117056,"data_stored":20582720,"data_compressed":386342,"data_compressed_allocated":1773568,"data_compressed_original":19795968,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640269,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43008,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328832,"statfs":{"total":96636764160,"available":96592723968,"internally_reserved":0,"allocated":3588096,"data_stored":24451923,"data_compressed":398456,"data_compressed_allocated":1855488,"data_compressed_original":23371264,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640268,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38496,"kb_used_data":3152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333344,"statfs":{"total":96636764160,"available":96597344256,"internally_reserved":0,"allocated":3227648,"data_stored":21479279,"data_compressed":344379,"data_compressed_allocated":1646592,"data_compressed_original":20530688,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640270,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60820,"kb_used_data":3264,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311020,"statfs":{"total":96636764160,"available":96574484480,"internally_reserved":0,"allocated":3342336,"data_stored":22135889,"data_compressed":314566,"data_compressed_allocated":1646592,"data_compressed_original":21105664,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640269,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39972,"kb_used_data":4628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331868,"statfs":{"total":96636764160,"available":96595832832,"internally_reserved":0,"allocated":4739072,"data_stored":32827330,"data_compressed":620129,"data_compressed_allocated":2863104,"data_compressed_original":31609856,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3743744,"data_stored":31171822,"data_compressed":605436,"data_compressed_allocated":2478080,"data_compressed_original":30225408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3121152,"data_stored":26137522,"data_compressed":369295,"data_compressed_allocated":1937408,"data_compressed_original":25299968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2711552,"data_stored":21278961,"data_compressed":344379,"data_compressed_allocated":1646592,"data_compressed_original":20530688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3022848,"data_stored":23666343,"data_compressed":389906,"data_compressed_allocated":1818624,"data_compressed_original":22797824,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2703360,"data_stored":24144052,"data_compressed":439126,"data_compressed_allocated":1892352,"data_compressed_original":23572480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:03:16.789 INFO:tasks.thrashosds.thrasher:Setting 3.16 to [3, 0] 2024-09-18T01:03:16.789 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.16', '3', '0'] 2024-09-18T01:03:16.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.16 3 0 2024-09-18T01:03:16.797 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:16.797 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:16.815 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:16.835 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:16.853 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:16.854 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:16.868 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:16.867+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:16.884 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:16.885 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:16.894 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:16.969 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:16.968+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:17.039 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.039 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.047 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:17.070 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:17.069+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:17.094 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-18T01:03:17.168 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.168 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.171 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:17.171+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:17.179 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:17.273 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:17.272+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:17.281 INFO:teuthology.orchestra.run.smithi164.stdout:2044 read 0x7b66800 thru 0x7b7216b (0xb96c bytes) 2024-09-18T01:03:17.282 INFO:teuthology.orchestra.run.smithi164.stdout:2045 read 0x53e0800 thru 0x53f05b3 (0xfdb4 bytes) 2024-09-18T01:03:17.283 INFO:teuthology.orchestra.run.smithi164.stdout:2047 punch from 0x734e600 to 0x735ca00, (0xe400 bytes) 2024-09-18T01:03:17.286 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.287 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.299 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:17.321 INFO:teuthology.orchestra.run.smithi164.stdout:2049 read 0xd2d000 thru 0xd3b058 (0xe059 bytes) 2024-09-18T01:03:17.324 INFO:teuthology.orchestra.run.smithi164.stdout:2053 trunc from 0x86c5200 to 0xa4b5e00 2024-09-18T01:03:17.336 INFO:teuthology.orchestra.run.smithi164.stdout:2054 punch from 0x80bdc00 to 0x80c7a00, (0x9e00 bytes) 2024-09-18T01:03:17.341 INFO:teuthology.orchestra.run.smithi164.stdout:2058 punch from 0x9632c00 to 0x9642600, (0xfa00 bytes) 2024-09-18T01:03:17.344 INFO:teuthology.orchestra.run.smithi164.stdout:2060 read 0x7621400 thru 0x7629f99 (0x8b9a bytes) 2024-09-18T01:03:17.345 INFO:teuthology.orchestra.run.smithi164.stdout:2061 write 0xa20f000 thru 0xa21a005 (0xb006 bytes) 2024-09-18T01:03:17.362 INFO:teuthology.orchestra.run.smithi164.stdout:2064 trunc from 0xa4b5e00 to 0x5f4b800 2024-09-18T01:03:17.374 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:17.373+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:17.444 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:03:17.448 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.16 pg_upmap mapping to [3,0] 2024-09-18T01:03:17.465 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.465 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:17.512 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:17.511+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:17.576 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:17.575+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:17.592 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.592 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.601 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:17.677 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:17.675+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:17.703 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.703 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:17.777 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:17.776+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:17.820 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.820 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:17.913 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:17.912+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:17.976 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:17.976 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:17.978 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:17.977+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:17.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:18.079 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:18.078+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:18.131 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:18.131 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:18.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:18.179 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:18.178+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:18.281 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:18.280+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:18.289 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:18.290 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:18.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:18.382 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:18.381+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:18.405 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:18.406 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:18.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:18.483 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:18.482+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:18.522 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:18.522 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:18.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:18.585 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:18.584+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:18.678 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:18.685 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:18.684+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:18.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:18.789 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:18.788+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:18.834 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:18.834 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:18.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:18.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:18.927 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:18.926+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:18.990 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:18.989+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:18.994 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:18.994 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:19.004 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:19.009 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:19.009 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:19.027 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:19.027 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:19.043 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:19.043 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:19.060 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:19.061 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:19.078 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:19.078 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:19.090 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:19.089+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:19.095 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:19.120 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:19.120 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:19.121 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:19.121 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:19.129 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:19.191 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:19.190+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:19.275 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:19.275 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:19.275 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:19.284 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:19.292 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:19.291+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:19.392 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:19.391+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:19.394 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:19.394 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:19.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:19.394 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:19.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.394 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.348976006000001, 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.395 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:03:19.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.399 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.348964854999998, 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:03:19.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.348953025, 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.405 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:03:19.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.410 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.348936342000002, 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:19.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.415 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.348925104999999, 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:03:19.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.347602999999999, 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:03:19.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.425 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.347590284999999, 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:03:19.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.347545301, 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.431 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:03:19.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.435 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.347517676999999, 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:03:19.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:03:19.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.343970297999999, 2024-09-18T01:03:19.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:19.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:03:19.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.445 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.343610191, 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:03:19.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.343584077999999, 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:03:19.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:03:19.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.455 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23940 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894162912] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.341414364999999, 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037903539999999999, 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23940 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:03:19.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051180+0000", 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4190000000000001e-06 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051183+0000", 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.774e-06 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051185+0000", 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5330000000000001e-06 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.088984+0000", 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037798867999999999 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089016+0000", 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2333999999999999e-05 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089082+0000", 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5399000000000002e-05 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.341404690000001, 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:03:19.460 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:03:19.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.341394895000001, 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.465 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.469 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.341384116, 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.470 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:03:19.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:19.474 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.34136432, 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:19.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:03:19.478 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.3395808, 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:03:19.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.483 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.339553962, 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:03:19.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 17.339543414000001, 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:03:19.488 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:03:19.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.492 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.493 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.493 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:19.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:03:19.495 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:19.495 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:19.493+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:19.549 INFO:teuthology.orchestra.run.smithi164.stdout:2065 write 0x11fc200 thru 0x120865a (0xc45b bytes) 2024-09-18T01:03:19.550 INFO:teuthology.orchestra.run.smithi164.stdout:2066 read 0xcb6a00 thru 0xcba5eb (0x3bec bytes) 2024-09-18T01:03:19.567 INFO:teuthology.orchestra.run.smithi164.stdout:2070 write 0x7cf1600 thru 0x7cf25d9 (0xfda bytes) 2024-09-18T01:03:19.569 INFO:teuthology.orchestra.run.smithi164.stdout:2071 write 0x6cfe400 thru 0x6d031b5 (0x4db6 bytes) 2024-09-18T01:03:19.569 INFO:teuthology.orchestra.run.smithi164.stdout:2072 write 0x57fc800 thru 0x57fd3cd (0xbce bytes) 2024-09-18T01:03:19.585 INFO:teuthology.orchestra.run.smithi164.stdout:2073 write 0xa6c6400 thru 0xa6d3736 (0xd337 bytes) 2024-09-18T01:03:19.592 INFO:teuthology.orchestra.run.smithi164.stdout:2074 trunc from 0xa6d3800 to 0x73dce00 2024-09-18T01:03:19.595 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:19.594+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:19.611 INFO:teuthology.orchestra.run.smithi164.stdout:2077 read 0x5cd6600 thru 0x5cdd8a3 (0x72a4 bytes) 2024-09-18T01:03:19.612 INFO:teuthology.orchestra.run.smithi164.stdout:2079 write 0x2e71600 thru 0x2e79f65 (0x8966 bytes) 2024-09-18T01:03:19.613 INFO:teuthology.orchestra.run.smithi164.stdout:2080 write 0x5ce0e00 thru 0x5ce2e9d (0x209e bytes) 2024-09-18T01:03:19.613 INFO:teuthology.orchestra.run.smithi164.stdout:2081 write 0x171ec00 thru 0x172b7bc (0xcbbd bytes) 2024-09-18T01:03:19.614 INFO:teuthology.orchestra.run.smithi164.stdout:2083 read 0x205e200 thru 0x2062b68 (0x4969 bytes) 2024-09-18T01:03:19.615 INFO:teuthology.orchestra.run.smithi164.stdout:2084 read 0x44ab000 thru 0x44b9b94 (0xeb95 bytes) 2024-09-18T01:03:19.619 INFO:teuthology.orchestra.run.smithi164.stdout:2085 trunc from 0x73dce00 to 0x521d000 2024-09-18T01:03:19.638 INFO:teuthology.orchestra.run.smithi164.stdout:2087 read 0x4c31a00 thru 0x4c39a78 (0x8079 bytes) 2024-09-18T01:03:19.646 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:19.646 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:19.646 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:19.646 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:19.654 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:19.656 INFO:teuthology.orchestra.run.smithi164.stdout:2089 write 0xa0c3400 thru 0xa0d3240 (0xfe41 bytes) 2024-09-18T01:03:19.667 INFO:teuthology.orchestra.run.smithi164.stdout:2090 trunc from 0xa0d3400 to 0x1c72800 2024-09-18T01:03:19.696 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:19.695+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:19.705 INFO:teuthology.orchestra.run.smithi164.stdout:2093 punch from 0x1427c00 to 0x1437200, (0xf600 bytes) 2024-09-18T01:03:19.726 INFO:teuthology.orchestra.run.smithi164.stdout:2096 write 0x85a2200 thru 0x85a67c7 (0x45c8 bytes) 2024-09-18T01:03:19.727 INFO:teuthology.orchestra.run.smithi164.stdout:2097 write 0x4732800 thru 0x47331c3 (0x9c4 bytes) 2024-09-18T01:03:19.727 INFO:teuthology.orchestra.run.smithi164.stdout:2100 trunc from 0x85a6800 to 0x83e8600 2024-09-18T01:03:19.760 INFO:teuthology.orchestra.run.smithi164.stdout:2101 write 0x95b1600 thru 0x95c15c4 (0xffc5 bytes) 2024-09-18T01:03:19.761 INFO:teuthology.orchestra.run.smithi164.stdout:2105 read 0x4832600 thru 0x48349fb (0x23fc bytes) 2024-09-18T01:03:19.782 INFO:teuthology.orchestra.run.smithi164.stdout:2107 write 0xc732800 thru 0xc73dfdd (0xb7de bytes) 2024-09-18T01:03:19.782 INFO:teuthology.orchestra.run.smithi164.stdout:2110 read 0x5c4f400 thru 0x5c5f1ed (0xfdee bytes) 2024-09-18T01:03:19.795 INFO:teuthology.orchestra.run.smithi164.stdout:2111 write 0xdef3c00 thru 0xdefd093 (0x9494 bytes) 2024-09-18T01:03:19.797 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:19.796+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:19.801 INFO:teuthology.orchestra.run.smithi164.stdout:2112 punch from 0x4a4ba00 to 0x4a4fe00, (0x4400 bytes) 2024-09-18T01:03:19.802 INFO:teuthology.orchestra.run.smithi164.stdout:2113 write 0x7a98e00 thru 0x7a9c0a7 (0x32a8 bytes) 2024-09-18T01:03:19.802 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:19.803 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:19.803 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:19.803 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:19.803 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:19.808 INFO:teuthology.orchestra.run.smithi164.stdout:2117 trunc from 0xdefd200 to 0xb5e7200 2024-09-18T01:03:19.812 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:19.821 INFO:teuthology.orchestra.run.smithi164.stdout:2118 trunc from 0xb5e7200 to 0xc0f6a00 2024-09-18T01:03:19.830 INFO:teuthology.orchestra.run.smithi164.stdout:2119 read 0x793bc00 thru 0x7948b4b (0xcf4c bytes) 2024-09-18T01:03:19.832 INFO:teuthology.orchestra.run.smithi164.stdout:2122 flatten 2024-09-18T01:03:19.846 INFO:teuthology.orchestra.run.smithi164.stdout:2123 trunc from 0xc0f6a00 to 0xac7b400 2024-09-18T01:03:19.857 INFO:teuthology.orchestra.run.smithi164.stdout:2124 write 0x6654000 thru 0x66576e8 (0x36e9 bytes) 2024-09-18T01:03:19.862 INFO:teuthology.orchestra.run.smithi164.stdout:2127 trunc from 0xac7b400 to 0x6c9a000 2024-09-18T01:03:19.884 INFO:teuthology.orchestra.run.smithi164.stdout:2129 trunc from 0x6c9a000 to 0x148de00 2024-09-18T01:03:19.898 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:19.897+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:19.915 INFO:teuthology.orchestra.run.smithi164.stdout:2130 punch from 0xe94a00 to 0xe9a600, (0x5c00 bytes) 2024-09-18T01:03:19.922 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:19.922 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624921168, 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.923 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:19.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624897051, 2024-09-18T01:03:19.927 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:19.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:03:19.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:03:19.931 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624817811, 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.932 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.933 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:03:19.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624781592, 2024-09-18T01:03:19.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:19.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624752979, 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:03:19.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:03:19.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624732114, 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.945 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.946 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:03:19.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:03:19.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:19.949 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624114397, 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:19.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.952 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624085129999999, 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:19.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.957 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.624018839, 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.958 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:03:19.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623898071999999, 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:03:19.962 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:03:19.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623679625999999, 2024-09-18T01:03:19.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:19.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:03:19.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623658265, 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.972 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:03:19.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623504326000001, 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.976 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:19.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:03:19.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:19.980 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623484608, 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:19.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623464298, 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:03:19.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623444412, 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.989 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:03:19.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623424453, 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:03:19.993 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:19.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623404454999999, 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.998 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:19.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:03:20.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623384787999999, 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:20.002 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:03:20.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.006 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 13.623364885999999, 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:03:20.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:03:20.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:03:20.012 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.012 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.012 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.012 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.012 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.013 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:20.014 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:20.013+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:20.015 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:20.016 INFO:teuthology.orchestra.run.smithi164.stdout:2132 read 0xfae00 thru 0xfb673 (0x874 bytes) 2024-09-18T01:03:20.016 INFO:teuthology.orchestra.run.smithi164.stdout:2133 punch from 0x795600 to 0x79d000, (0x7a00 bytes) 2024-09-18T01:03:20.016 INFO:teuthology.orchestra.run.smithi164.stdout:2134 trunc from 0x148de00 to 0x3614800 2024-09-18T01:03:20.016 INFO:teuthology.orchestra.run.smithi164.stdout:2136 write 0x16e6200 thru 0x16f40e5 (0xdee6 bytes) 2024-09-18T01:03:20.016 INFO:teuthology.orchestra.run.smithi164.stdout:2140 trunc from 0x3614800 to 0x7794000 2024-09-18T01:03:20.016 INFO:teuthology.orchestra.run.smithi164.stdout:2143 punch from 0x18f2000 to 0x18fee00, (0xce00 bytes) 2024-09-18T01:03:20.017 INFO:teuthology.orchestra.run.smithi164.stdout:2144 read 0x36bb400 thru 0x36bb4e6 (0xe7 bytes) 2024-09-18T01:03:20.017 INFO:teuthology.orchestra.run.smithi164.stdout:2145 trunc from 0x7794000 to 0x1034600 2024-09-18T01:03:20.017 INFO:teuthology.orchestra.run.smithi164.stdout:2146 write 0x429d000 thru 0x42aac8d (0xdc8e bytes) 2024-09-18T01:03:20.017 INFO:teuthology.orchestra.run.smithi164.stdout:2148 read 0x65d600 thru 0x665c2c (0x862d bytes) 2024-09-18T01:03:20.017 INFO:teuthology.orchestra.run.smithi164.stdout:2150 write 0xd33de00 thru 0xd34c166 (0xe367 bytes) 2024-09-18T01:03:20.030 INFO:teuthology.orchestra.run.smithi164.stdout:2151 write 0xd68fa00 thru 0xd69b29e (0xb89f bytes) 2024-09-18T01:03:20.031 INFO:teuthology.orchestra.run.smithi164.stdout:2152 read 0x9569200 thru 0x9569e3e (0xc3f bytes) 2024-09-18T01:03:20.031 INFO:teuthology.orchestra.run.smithi164.stdout:2154 read 0x23e4800 thru 0x23e81cf (0x39d0 bytes) 2024-09-18T01:03:20.031 INFO:teuthology.orchestra.run.smithi164.stdout:2157 read 0x806aa00 thru 0x807801c (0xd61d bytes) 2024-09-18T01:03:20.031 INFO:teuthology.orchestra.run.smithi164.stdout:2158 read 0x4b16e00 thru 0x4b23513 (0xc714 bytes) 2024-09-18T01:03:20.031 INFO:teuthology.orchestra.run.smithi164.stdout:2160 read 0xb646600 thru 0xb64f0a8 (0x8aa9 bytes) 2024-09-18T01:03:20.032 INFO:teuthology.orchestra.run.smithi164.stdout:2161 punch from 0xa590400 to 0xa59aa00, (0xa600 bytes) 2024-09-18T01:03:20.037 INFO:teuthology.orchestra.run.smithi164.stdout:2164 trunc from 0xd69b400 to 0x95b9200 2024-09-18T01:03:20.049 INFO:teuthology.orchestra.run.smithi164.stdout:2165 read 0x4b04c00 thru 0x4b147ab (0xfbac bytes) 2024-09-18T01:03:20.050 INFO:teuthology.orchestra.run.smithi164.stdout:2169 punch from 0x2714600 to 0x2720200, (0xbc00 bytes) 2024-09-18T01:03:20.051 INFO:teuthology.orchestra.run.smithi164.stdout:2170 write 0x2907c00 thru 0x290cef4 (0x52f5 bytes) 2024-09-18T01:03:20.051 INFO:teuthology.orchestra.run.smithi164.stdout:2172 write 0x3753200 thru 0x375ba3d (0x883e bytes) 2024-09-18T01:03:20.052 INFO:teuthology.orchestra.run.smithi164.stdout:2174 read 0x67b6600 thru 0x67b852a (0x1f2b bytes) 2024-09-18T01:03:20.052 INFO:teuthology.orchestra.run.smithi164.stdout:2175 read 0x2be0000 thru 0x2be7c50 (0x7c51 bytes) 2024-09-18T01:03:20.052 INFO:teuthology.orchestra.run.smithi164.stdout:2177 write 0xbd9c00 thru 0xbe5c05 (0xc006 bytes) 2024-09-18T01:03:20.053 INFO:teuthology.orchestra.run.smithi164.stdout:2179 read 0x529ae00 thru 0x52a1154 (0x6355 bytes) 2024-09-18T01:03:20.074 INFO:teuthology.orchestra.run.smithi164.stdout:2182 write 0xc6f5800 thru 0xc7010e2 (0xb8e3 bytes) 2024-09-18T01:03:20.076 INFO:teuthology.orchestra.run.smithi164.stdout:2183 punch from 0x7308800 to 0x730ae00, (0x2600 bytes) 2024-09-18T01:03:20.079 INFO:teuthology.orchestra.run.smithi164.stdout:2184 write 0x3ad3a00 thru 0x3ae080c (0xce0d bytes) 2024-09-18T01:03:20.079 INFO:teuthology.orchestra.run.smithi164.stdout:2186 read 0x1ef8a00 thru 0x1f063dc (0xd9dd bytes) 2024-09-18T01:03:20.079 INFO:teuthology.orchestra.run.smithi164.stdout:2187 write 0x56ae200 thru 0x56b1408 (0x3209 bytes) 2024-09-18T01:03:20.080 INFO:teuthology.orchestra.run.smithi164.stdout:2189 read 0xbe87000 thru 0xbe8dcb3 (0x6cb4 bytes) 2024-09-18T01:03:20.082 INFO:teuthology.orchestra.run.smithi164.stdout:2190 trunc from 0xc701200 to 0xe213600 2024-09-18T01:03:20.094 INFO:teuthology.orchestra.run.smithi164.stdout:2192 read 0x5835400 thru 0x583b0cb (0x5ccc bytes) 2024-09-18T01:03:20.110 INFO:teuthology.orchestra.run.smithi164.stdout:2194 trunc from 0xe213600 to 0x177ae00 2024-09-18T01:03:20.115 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:20.113+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:20.125 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:20.125 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:20.135 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:20.136 INFO:teuthology.orchestra.run.smithi164.stdout:2195 read 0x9cbc00 thru 0x9cd450 (0x1851 bytes) 2024-09-18T01:03:20.143 INFO:teuthology.orchestra.run.smithi164.stdout:2196 trunc from 0x177ae00 to 0x534a600 2024-09-18T01:03:20.152 INFO:teuthology.orchestra.run.smithi164.stdout:2200 read 0x2354a00 thru 0x235dd73 (0x9374 bytes) 2024-09-18T01:03:20.153 INFO:teuthology.orchestra.run.smithi164.stdout:2201 read 0x47e6a00 thru 0x47f3e23 (0xd424 bytes) 2024-09-18T01:03:20.165 INFO:teuthology.orchestra.run.smithi164.stdout:2202 write 0x7289e00 thru 0x7297199 (0xd39a bytes) 2024-09-18T01:03:20.177 INFO:teuthology.orchestra.run.smithi164.stdout:2203 write 0x8b06400 thru 0x8b092fc (0x2efd bytes) 2024-09-18T01:03:20.178 INFO:teuthology.orchestra.run.smithi164.stdout:2204 read 0x654be00 thru 0x655208b (0x628c bytes) 2024-09-18T01:03:20.179 INFO:teuthology.orchestra.run.smithi164.stdout:2206 read 0x8116e00 thru 0x8121743 (0xa944 bytes) 2024-09-18T01:03:20.179 INFO:teuthology.orchestra.run.smithi164.stdout:2207 punch from 0x2165800 to 0x216b600, (0x5e00 bytes) 2024-09-18T01:03:20.180 INFO:teuthology.orchestra.run.smithi164.stdout:2208 write 0x26c9400 thru 0x26d5749 (0xc34a bytes) 2024-09-18T01:03:20.181 INFO:teuthology.orchestra.run.smithi164.stdout:2210 trunc from 0x8b09400 to 0x9742000 2024-09-18T01:03:20.193 INFO:teuthology.orchestra.run.smithi164.stdout:2211 read 0x1cd7800 thru 0x1cdd029 (0x582a bytes) 2024-09-18T01:03:20.193 INFO:teuthology.orchestra.run.smithi164.stdout:2212 read 0x87f3800 thru 0x87f6992 (0x3193 bytes) 2024-09-18T01:03:20.197 INFO:teuthology.orchestra.run.smithi164.stdout:2216 trunc from 0x9742000 to 0x6662600 2024-09-18T01:03:20.210 INFO:teuthology.orchestra.run.smithi164.stdout:2217 write 0x6ac400 thru 0x6ba51c (0xe11d bytes) 2024-09-18T01:03:20.211 INFO:teuthology.orchestra.run.smithi164.stdout:2218 read 0x16f0000 thru 0x16f9653 (0x9654 bytes) 2024-09-18T01:03:20.211 INFO:teuthology.orchestra.run.smithi164.stdout:2219 punch from 0x2ef9800 to 0x2efc000, (0x2800 bytes) 2024-09-18T01:03:20.212 INFO:teuthology.orchestra.run.smithi164.stdout:2221 punch from 0xb08400 to 0xb0b000, (0x2c00 bytes) 2024-09-18T01:03:20.220 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:20.219+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:20.221 INFO:teuthology.orchestra.run.smithi164.stdout:2223 trunc from 0x6662600 to 0xe5e3400 2024-09-18T01:03:20.230 INFO:teuthology.orchestra.run.smithi164.stdout:2230 write 0x61e3a00 thru 0x61e7c25 (0x4226 bytes) 2024-09-18T01:03:20.239 INFO:teuthology.orchestra.run.smithi164.stdout:2232 read 0x941f600 thru 0x9425c6b (0x666c bytes) 2024-09-18T01:03:20.239 INFO:teuthology.orchestra.run.smithi164.stdout:2233 read 0x7206a00 thru 0x72100c2 (0x96c3 bytes) 2024-09-18T01:03:20.240 INFO:teuthology.orchestra.run.smithi164.stdout:2234 write 0x1224600 thru 0x1232819 (0xe21a bytes) 2024-09-18T01:03:20.240 INFO:teuthology.orchestra.run.smithi164.stdout:2237 clone 15 order 23 su 32768 sc 12 2024-09-18T01:03:20.281 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:20.281 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:20.281 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:20.281 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:20.281 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:03:20.290 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:20.331 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:20.330+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:20.421 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:20.416+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:20.436 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:20.436 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:20.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:20.436 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:20.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.436 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:20.436 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 48.059870259, 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:20.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:20.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.061858954000002, 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.444 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:20.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:20.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:20.451 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 46.054182001999997, 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.452 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:20.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:20.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:20.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.460 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:20.460 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:20.460 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.956324051999999, 2024-09-18T01:03:20.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:20.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:20.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.467 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.100243040000002, 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.468 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:20.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:20.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:20.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.099873744, 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.098134899000001, 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.098063009000001, 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.010196999000001, 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.004005739999997, 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.002476055000002, 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 33.002096366000004, 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.986597963000001, 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.986391204, 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.985973446999999, 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.985764209000003, 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.985700274999999, 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.984952069999999, 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.726013264999999, 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 32.725149647000002, 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:20.558 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.559 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.559 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.559 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:20.559 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:20.559 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:20.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:20.561 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:20.559+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:20.661 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:20.660+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:20.674 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:20.674 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:20.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:20.762 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:20.761+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:20.794 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:20.794 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:20.794 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:20.794 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:20.794 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:20.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:20.833 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-18T01:03:20.887 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:20.886+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:20.952 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2015 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4265984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.631698704000001, 2024-09-18T01:03:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027955569999999999, 2024-09-18T01:03:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2015 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318446+0000", 2024-09-18T01:03:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318453+0000", 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318458+0000", 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9910000000000002e-06 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346240+0000", 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027782305 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346316+0000", 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5731999999999999e-05 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346399+0000", 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3137e-05 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.631677803999999, 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.631657295, 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:20.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.631637489999999, 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.63160285, 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:20.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:20.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.631581349000001, 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:20.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.631558171999998, 2024-09-18T01:03:20.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:20.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2237 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4202496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627589387, 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028056793, 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:20.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2237 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322555+0000", 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7740000000000001e-06 2024-09-18T01:03:20.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322562+0000", 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.426e-06 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322567+0000", 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:20.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350490+0000", 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027923436999999999 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350528+0000", 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7660999999999997e-05 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350610+0000", 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2150000000000005e-05 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:20.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627569340000001, 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:21.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:21.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627549787, 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:21.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627529283000001, 2024-09-18T01:03:21.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:21.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:21.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627508419000002, 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:21.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627488519, 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:21.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627464431, 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:21.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:21.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:21.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:21.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627433110999998, 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:21.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:21.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627388504999999, 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:21.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627357713999999, 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:21.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627334228999999, 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:21.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:21.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:21.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:21.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627305174, 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:21.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:21.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.627284414999998, 2024-09-18T01:03:21.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:21.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:21.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:21.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:21.063 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:21.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:21.065 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:21.063+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:21.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:21.165 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:21.164+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:21.217 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:21.217 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:21.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:21.231 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:21.231 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:21.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:21.266 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:21.267 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:21.267 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:21.265+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:21.284 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:21.284 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:21.307 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:21.307 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:21.339 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:21.340 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:21.371 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:21.371 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:21.371 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:21.370+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:21.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:21.468 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:21.467+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:21.535 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:21.535 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:21.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:21.588 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:21.587+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:21.670 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:21.668+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:21.673 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:21.673 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:21.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:21.770 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:21.769+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:21.838 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:21.838 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:21.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:21.872 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:21.870+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:21.973 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:21.972+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:22.028 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:22.028 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:22.040 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:22.081 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:22.080+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:22.168 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:22.168 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:22.175 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:22.174+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:22.181 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:22.275 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:22.274+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:22.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:03:22.308 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:22.309 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:22.318 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:22.403 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:22.402+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:22.461 INFO:teuthology.orchestra.run.smithi164.stdout:2239 punch from 0x721f000 to 0x7225400, (0x6400 bytes) 2024-09-18T01:03:22.461 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:22.461 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:22.471 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:22.471 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:22.471 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-18T01:03:22.471 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:03:22.471 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:03:22.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-18T01:03:22.473 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:22.482 INFO:teuthology.orchestra.run.smithi164.stdout:2240 write 0xd407400 thru 0xd4159c9 (0xe5ca bytes) 2024-09-18T01:03:22.483 INFO:teuthology.orchestra.run.smithi164.stdout:2241 read 0x1ac0000 thru 0x1ace967 (0xe968 bytes) 2024-09-18T01:03:22.483 INFO:teuthology.orchestra.run.smithi164.stdout:2242 punch from 0xcdb0e00 to 0xcdb5c00, (0x4e00 bytes) 2024-09-18T01:03:22.514 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:22.513+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:22.516 INFO:teuthology.orchestra.run.smithi164.stdout:2245 trunc from 0xe5e3400 to 0x8bae00 2024-09-18T01:03:22.578 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:22.577+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:22.630 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:22.630 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:03:22.630 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:22.641 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:22.641 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:22.652 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:22.677 INFO:teuthology.orchestra.run.smithi164.stdout:2249 read 0x843a00 thru 0x84e2b6 (0xa8b7 bytes) 2024-09-18T01:03:22.695 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:22.694+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:22.701 INFO:teuthology.orchestra.run.smithi164.stdout:2250 write 0xdb54a00 thru 0xdb60365 (0xb966 bytes) 2024-09-18T01:03:22.702 INFO:teuthology.orchestra.run.smithi164.stdout:2252 write 0x1a65a00 thru 0x1a69489 (0x3a8a bytes) 2024-09-18T01:03:22.704 INFO:teuthology.orchestra.run.smithi164.stdout:2253 write 0x73b0400 thru 0x73bd059 (0xcc5a bytes) 2024-09-18T01:03:22.704 INFO:teuthology.orchestra.run.smithi164.stdout:2255 write 0xb3e6a00 thru 0xb3ed218 (0x6819 bytes) 2024-09-18T01:03:22.704 INFO:teuthology.orchestra.run.smithi164.stdout:2256 write 0x3a39e00 thru 0x3a444ad (0xa6ae bytes) 2024-09-18T01:03:22.704 INFO:teuthology.orchestra.run.smithi164.stdout:2258 read 0x6ffb400 thru 0x700859e (0xd19f bytes) 2024-09-18T01:03:22.705 INFO:teuthology.orchestra.run.smithi164.stdout:2260 write 0x4d2cc00 thru 0x4d2e795 (0x1b96 bytes) 2024-09-18T01:03:22.705 INFO:teuthology.orchestra.run.smithi164.stdout:2261 read 0xa14ae00 thru 0xa152eac (0x80ad bytes) 2024-09-18T01:03:22.705 INFO:teuthology.orchestra.run.smithi164.stdout:2262 write 0x2b16600 thru 0x2b17119 (0xb1a bytes) 2024-09-18T01:03:22.705 INFO:teuthology.orchestra.run.smithi164.stdout:2263 write 0x6062000 thru 0x606eae1 (0xcae2 bytes) 2024-09-18T01:03:22.705 INFO:teuthology.orchestra.run.smithi164.stdout:2265 read 0x3ba9e00 thru 0x3bb6ea3 (0xd0a4 bytes) 2024-09-18T01:03:22.706 INFO:teuthology.orchestra.run.smithi164.stdout:2266 write 0xd4de600 thru 0xd4ee0ca (0xfacb bytes) 2024-09-18T01:03:22.706 INFO:teuthology.orchestra.run.smithi164.stdout:2267 punch from 0x84a4c00 to 0x84b0000, (0xb400 bytes) 2024-09-18T01:03:22.707 INFO:teuthology.orchestra.run.smithi164.stdout:2270 write 0xd176200 thru 0xd17e01c (0x7e1d bytes) 2024-09-18T01:03:22.707 INFO:teuthology.orchestra.run.smithi164.stdout:2271 write 0x74ba800 thru 0x74bb696 (0xe97 bytes) 2024-09-18T01:03:22.732 INFO:teuthology.orchestra.run.smithi164.stdout:2272 write 0xe8e4e00 thru 0xe8f0034 (0xb235 bytes) 2024-09-18T01:03:22.733 INFO:teuthology.orchestra.run.smithi164.stdout:2273 read 0x3f7a00 thru 0x3f98a0 (0x1ea1 bytes) 2024-09-18T01:03:22.734 INFO:teuthology.orchestra.run.smithi164.stdout:2274 write 0x5687a00 thru 0x568aae0 (0x30e1 bytes) 2024-09-18T01:03:22.735 INFO:teuthology.orchestra.run.smithi164.stdout:2276 write 0xa296800 thru 0xa2a40d2 (0xd8d3 bytes) 2024-09-18T01:03:22.735 INFO:teuthology.orchestra.run.smithi164.stdout:2277 write 0x3498400 thru 0x349c417 (0x4018 bytes) 2024-09-18T01:03:22.735 INFO:teuthology.orchestra.run.smithi164.stdout:2280 write 0x2574800 thru 0x257d919 (0x911a bytes) 2024-09-18T01:03:22.736 INFO:teuthology.orchestra.run.smithi164.stdout:2282 write 0xe623200 thru 0xe62db10 (0xa911 bytes) 2024-09-18T01:03:22.753 INFO:teuthology.orchestra.run.smithi164.stdout:2284 write 0xed02a00 thru 0xed11a7a (0xf07b bytes) 2024-09-18T01:03:22.755 INFO:teuthology.orchestra.run.smithi164.stdout:2285 read 0x711f800 thru 0x712b5a8 (0xbda9 bytes) 2024-09-18T01:03:22.761 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:22.761 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:22.767 INFO:teuthology.orchestra.run.smithi164.stdout:2287 trunc from 0xed11c00 to 0x5a17600 2024-09-18T01:03:22.770 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:22.780 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:22.779+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:22.794 INFO:teuthology.orchestra.run.smithi164.stdout:2289 read 0x23e7200 thru 0x23f10ab (0x9eac bytes) 2024-09-18T01:03:22.801 INFO:teuthology.orchestra.run.smithi164.stdout:2291 read 0x3faec00 thru 0x3fb0f7d (0x237e bytes) 2024-09-18T01:03:22.805 INFO:teuthology.orchestra.run.smithi164.stdout:2294 trunc from 0x5a17600 to 0x8f90a00 2024-09-18T01:03:22.818 INFO:teuthology.orchestra.run.smithi164.stdout:2296 read 0x13b0000 thru 0x13bdc84 (0xdc85 bytes) 2024-09-18T01:03:22.819 INFO:teuthology.orchestra.run.smithi164.stdout:2297 punch from 0x180ae00 to 0x180c000, (0x1200 bytes) 2024-09-18T01:03:22.820 INFO:teuthology.orchestra.run.smithi164.stdout:2298 read 0x29bb000 thru 0x29c372b (0x872c bytes) 2024-09-18T01:03:22.820 INFO:teuthology.orchestra.run.smithi164.stdout:2299 punch from 0x2477400 to 0x2482400, (0xb000 bytes) 2024-09-18T01:03:22.820 INFO:teuthology.orchestra.run.smithi164.stdout:2300 write 0x48f400 thru 0x495a4b (0x664c bytes) 2024-09-18T01:03:22.840 INFO:teuthology.orchestra.run.smithi164.stdout:2301 write 0xa853000 thru 0xa85ef33 (0xbf34 bytes) 2024-09-18T01:03:22.840 INFO:teuthology.orchestra.run.smithi164.stdout:2305 read 0x17f6800 thru 0x17fe6dd (0x7ede bytes) 2024-09-18T01:03:22.841 INFO:teuthology.orchestra.run.smithi164.stdout:2306 read 0x88a6800 thru 0x88a68c9 (0xca bytes) 2024-09-18T01:03:22.841 INFO:teuthology.orchestra.run.smithi164.stdout:2307 write 0x93f9000 thru 0x9401085 (0x8086 bytes) 2024-09-18T01:03:22.841 INFO:teuthology.orchestra.run.smithi164.stdout:2308 punch from 0x764bc00 to 0x765a800, (0xec00 bytes) 2024-09-18T01:03:22.841 INFO:teuthology.orchestra.run.smithi164.stdout:2309 read 0xa398600 thru 0xa3a4487 (0xbe88 bytes) 2024-09-18T01:03:22.841 INFO:teuthology.orchestra.run.smithi164.stdout:2310 write 0xa380c00 thru 0xa38de19 (0xd21a bytes) 2024-09-18T01:03:22.842 INFO:teuthology.orchestra.run.smithi164.stdout:2311 trunc from 0xa85f000 to 0xbe37600 2024-09-18T01:03:22.867 INFO:teuthology.orchestra.run.smithi164.stdout:2312 write 0x4291400 thru 0x4292630 (0x1231 bytes) 2024-09-18T01:03:22.869 INFO:teuthology.orchestra.run.smithi164.stdout:2313 trunc from 0xbe37600 to 0xb72e800 2024-09-18T01:03:22.881 INFO:teuthology.orchestra.run.smithi164.stdout:2314 read 0xe71a00 thru 0xe7ac24 (0x9225 bytes) 2024-09-18T01:03:22.882 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:22.880+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:22.889 INFO:teuthology.orchestra.run.smithi164.stdout:2322 read 0x26d6200 thru 0x26e196b (0xb76c bytes) 2024-09-18T01:03:22.889 INFO:teuthology.orchestra.run.smithi164.stdout:2324 read 0x91fa00 thru 0x92e5f1 (0xebf2 bytes) 2024-09-18T01:03:22.890 INFO:teuthology.orchestra.run.smithi164.stdout:2328 read 0x5cd5800 thru 0x5cdde1b (0x861c bytes) 2024-09-18T01:03:22.896 INFO:teuthology.orchestra.run.smithi164.stdout:2329 trunc from 0xb72e800 to 0x6568000 2024-09-18T01:03:22.909 INFO:teuthology.orchestra.run.smithi164.stdout:2330 punch from 0x5a20a00 to 0x5a2ec00, (0xe200 bytes) 2024-09-18T01:03:22.919 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:22.919 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:22.927 INFO:teuthology.orchestra.run.smithi164.stdout:2333 write 0x7fb6a00 thru 0x7fc0305 (0x9906 bytes) 2024-09-18T01:03:22.928 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:22.933 INFO:teuthology.orchestra.run.smithi164.stdout:2336 write 0x7de6800 thru 0x7df5e83 (0xf684 bytes) 2024-09-18T01:03:22.933 INFO:teuthology.orchestra.run.smithi164.stdout:2337 punch from 0x3636800 to 0x3637600, (0xe00 bytes) 2024-09-18T01:03:22.934 INFO:teuthology.orchestra.run.smithi164.stdout:2344 punch from 0x28cce00 to 0x28cfe00, (0x3000 bytes) 2024-09-18T01:03:22.934 INFO:teuthology.orchestra.run.smithi164.stdout:2346 punch from 0x1558600 to 0x1560e00, (0x8800 bytes) 2024-09-18T01:03:22.934 INFO:teuthology.orchestra.run.smithi164.stdout:2347 punch from 0x25a1200 to 0x25a5800, (0x4600 bytes) 2024-09-18T01:03:22.934 INFO:teuthology.orchestra.run.smithi164.stdout:2349 write 0x3e52200 thru 0x3e5731f (0x5120 bytes) 2024-09-18T01:03:22.936 INFO:teuthology.orchestra.run.smithi164.stdout:2354 punch from 0x20800 to 0x21a00, (0x1200 bytes) 2024-09-18T01:03:22.936 INFO:teuthology.orchestra.run.smithi164.stdout:2356 punch from 0x3547800 to 0x3557400, (0xfc00 bytes) 2024-09-18T01:03:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:2358 write 0xdcc4e00 thru 0xdcca5f6 (0x57f7 bytes) 2024-09-18T01:03:22.953 INFO:teuthology.orchestra.run.smithi164.stdout:2361 write 0x67a9c00 thru 0x67b30d7 (0x94d8 bytes) 2024-09-18T01:03:22.963 INFO:teuthology.orchestra.run.smithi164.stdout:2365 trunc from 0xdcca600 to 0x6a5ce00 2024-09-18T01:03:23.005 INFO:teuthology.orchestra.run.smithi164.stdout:2366 write 0xe8b2400 thru 0xe8b3c57 (0x1858 bytes) 2024-09-18T01:03:23.005 INFO:teuthology.orchestra.run.smithi164.stdout:2367 punch from 0xb928800 to 0xb92a800, (0x2000 bytes) 2024-09-18T01:03:23.006 INFO:teuthology.orchestra.run.smithi164.stdout:2369 read 0x3045000 thru 0x304b92f (0x6930 bytes) 2024-09-18T01:03:23.007 INFO:teuthology.orchestra.run.smithi164.stdout:2370 read 0xaf43400 thru 0xaf4439c (0xf9d bytes) 2024-09-18T01:03:23.007 INFO:teuthology.orchestra.run.smithi164.stdout:2373 write 0xcb6a800 thru 0xcb6b06f (0x870 bytes) 2024-09-18T01:03:23.007 INFO:teuthology.orchestra.run.smithi164.stdout:2374 write 0xe874c00 thru 0xe87f218 (0xa619 bytes) 2024-09-18T01:03:23.007 INFO:teuthology.orchestra.run.smithi164.stdout:2377 write 0xb9d3400 thru 0xb9d88f1 (0x54f2 bytes) 2024-09-18T01:03:23.008 INFO:teuthology.orchestra.run.smithi164.stdout:2379 punch from 0xf9d400 to 0xfab400, (0xe000 bytes) 2024-09-18T01:03:23.008 INFO:teuthology.orchestra.run.smithi164.stdout:2380 punch from 0x37d000 to 0x37e400, (0x1400 bytes) 2024-09-18T01:03:23.008 INFO:teuthology.orchestra.run.smithi164.stdout:2381 write 0xe7a9000 thru 0xe7ada92 (0x4a93 bytes) 2024-09-18T01:03:23.009 INFO:teuthology.orchestra.run.smithi164.stdout:2382 flatten 2024-09-18T01:03:23.015 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:23.014+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:23.041 INFO:teuthology.orchestra.run.smithi164.stdout:2383 trunc from 0xe8b3e00 to 0xc30600 2024-09-18T01:03:23.080 INFO:teuthology.orchestra.run.smithi164.stdout:2385 write 0x8a1ee00 thru 0x8a1fdb5 (0xfb6 bytes) 2024-09-18T01:03:23.081 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:23.081 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:23.081 INFO:teuthology.orchestra.run.smithi164.stdout:2387 read 0x53c7e00 thru 0x53d7df9 (0xfffa bytes) 2024-09-18T01:03:23.082 INFO:teuthology.orchestra.run.smithi164.stdout:2388 read 0x86d1600 thru 0x86d2a3a (0x143b bytes) 2024-09-18T01:03:23.082 INFO:teuthology.orchestra.run.smithi164.stdout:2389 trunc from 0x8a1fe00 to 0x8911c00 2024-09-18T01:03:23.085 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:23.084+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:23.091 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:23.096 INFO:teuthology.orchestra.run.smithi164.stdout:2390 punch from 0x3389a00 to 0x3390e00, (0x7400 bytes) 2024-09-18T01:03:23.102 INFO:teuthology.orchestra.run.smithi164.stdout:2391 write 0x5497000 thru 0x54a6cef (0xfcf0 bytes) 2024-09-18T01:03:23.102 INFO:teuthology.orchestra.run.smithi164.stdout:2392 read 0x6985600 thru 0x69888a3 (0x32a4 bytes) 2024-09-18T01:03:23.113 INFO:teuthology.orchestra.run.smithi164.stdout:2394 write 0x8f7e000 thru 0x8f87b0e (0x9b0f bytes) 2024-09-18T01:03:23.119 INFO:teuthology.orchestra.run.smithi164.stdout:2395 write 0x1a7c000 thru 0x1a7c3d6 (0x3d7 bytes) 2024-09-18T01:03:23.119 INFO:teuthology.orchestra.run.smithi164.stdout:2397 write 0x58c5c00 thru 0x58d3f3f (0xe340 bytes) 2024-09-18T01:03:23.140 INFO:teuthology.orchestra.run.smithi164.stdout:2398 write 0xcc54800 thru 0xcc60e12 (0xc613 bytes) 2024-09-18T01:03:23.144 INFO:teuthology.orchestra.run.smithi164.stdout:2399 trunc from 0xcc61000 to 0xb3a0200 2024-09-18T01:03:23.170 INFO:teuthology.orchestra.run.smithi164.stdout:2400 trunc from 0xb3a0200 to 0x48e1600 2024-09-18T01:03:23.186 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:23.185+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:23.195 INFO:teuthology.orchestra.run.smithi164.stdout:2403 trunc from 0x48e1600 to 0xb070e00 2024-09-18T01:03:23.210 INFO:teuthology.orchestra.run.smithi164.stdout:2404 trunc from 0xb070e00 to 0x53a2800 2024-09-18T01:03:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:23.245 INFO:teuthology.orchestra.run.smithi164.stdout:2410 write 0xe905400 thru 0xe90e54d (0x914e bytes) 2024-09-18T01:03:23.246 INFO:teuthology.orchestra.run.smithi164.stdout:2411 read 0xa900800 thru 0xa9040a1 (0x38a2 bytes) 2024-09-18T01:03:23.246 INFO:teuthology.orchestra.run.smithi164.stdout:2412 punch from 0x9c36c00 to 0x9c3ba00, (0x4e00 bytes) 2024-09-18T01:03:23.247 INFO:teuthology.orchestra.run.smithi164.stdout:2415 read 0x6737a00 thru 0x674347d (0xba7e bytes) 2024-09-18T01:03:23.247 INFO:teuthology.orchestra.run.smithi164.stdout:2416 write 0x1e9200 thru 0x1f696c (0xd76d bytes) 2024-09-18T01:03:23.248 INFO:teuthology.orchestra.run.smithi164.stdout:2421 write 0x729e00 thru 0x73474f (0xa950 bytes) 2024-09-18T01:03:23.251 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:23.252 INFO:teuthology.orchestra.run.smithi164.stdout:2423 read 0x4bf5a00 thru 0x4bfe2b1 (0x88b2 bytes) 2024-09-18T01:03:23.252 INFO:teuthology.orchestra.run.smithi164.stdout:2424 punch from 0x7341800 to 0x734f000, (0xd800 bytes) 2024-09-18T01:03:23.252 INFO:teuthology.orchestra.run.smithi164.stdout:2425 write 0xb5bde00 thru 0xb5bed45 (0xf46 bytes) 2024-09-18T01:03:23.252 INFO:teuthology.orchestra.run.smithi164.stdout:2427 read 0x87f7400 thru 0x87fe884 (0x7485 bytes) 2024-09-18T01:03:23.253 INFO:teuthology.orchestra.run.smithi164.stdout:2428 read 0x1c6d200 thru 0x1c79f48 (0xcd49 bytes) 2024-09-18T01:03:23.254 INFO:teuthology.orchestra.run.smithi164.stdout:2429 punch from 0xa7a0200 to 0xa7a0400, (0x200 bytes) 2024-09-18T01:03:23.255 INFO:teuthology.orchestra.run.smithi164.stdout:2430 punch from 0x3e7ce00 to 0x3e87600, (0xa800 bytes) 2024-09-18T01:03:23.255 INFO:teuthology.orchestra.run.smithi164.stdout:2431 punch from 0xa286a00 to 0xa294000, (0xd600 bytes) 2024-09-18T01:03:23.255 INFO:teuthology.orchestra.run.smithi164.stdout:2432 read 0x9f2be00 thru 0x9f2dc7d (0x1e7e bytes) 2024-09-18T01:03:23.255 INFO:teuthology.orchestra.run.smithi164.stdout:2433 write 0x2347800 thru 0x2351061 (0x9862 bytes) 2024-09-18T01:03:23.257 INFO:teuthology.orchestra.run.smithi164.stdout:2434 write 0xd497600 thru 0xd4a6907 (0xf308 bytes) 2024-09-18T01:03:23.257 INFO:teuthology.orchestra.run.smithi164.stdout:2435 read 0xd834800 thru 0xd837d81 (0x3582 bytes) 2024-09-18T01:03:23.257 INFO:teuthology.orchestra.run.smithi164.stdout:2440 punch from 0x8c32e00 to 0x8c3fc00, (0xce00 bytes) 2024-09-18T01:03:23.257 INFO:teuthology.orchestra.run.smithi164.stdout:2441 trunc from 0xe90e600 to 0xec11e00 2024-09-18T01:03:23.281 INFO:teuthology.orchestra.run.smithi164.stdout:2442 trunc from 0xec11e00 to 0xc9a0c00 2024-09-18T01:03:23.294 INFO:teuthology.orchestra.run.smithi164.stdout:2443 punch from 0x61a7e00 to 0x61b5800, (0xda00 bytes) 2024-09-18T01:03:23.295 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:23.294+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:23.295 INFO:teuthology.orchestra.run.smithi164.stdout:2444 write 0x2c72400 thru 0x2c7409a (0x1c9b bytes) 2024-09-18T01:03:23.296 INFO:teuthology.orchestra.run.smithi164.stdout:2445 read 0x1c9d400 thru 0x1ca76a6 (0xa2a7 bytes) 2024-09-18T01:03:23.297 INFO:teuthology.orchestra.run.smithi164.stdout:2446 punch from 0xb239400 to 0xb247600, (0xe200 bytes) 2024-09-18T01:03:23.297 INFO:teuthology.orchestra.run.smithi164.stdout:2448 write 0x3d95c00 thru 0x3d9eac4 (0x8ec5 bytes) 2024-09-18T01:03:23.297 INFO:teuthology.orchestra.run.smithi164.stdout:2449 write 0xbca2a00 thru 0xbca7e35 (0x5436 bytes) 2024-09-18T01:03:23.298 INFO:teuthology.orchestra.run.smithi164.stdout:2450 punch from 0x858c000 to 0x859bc00, (0xfc00 bytes) 2024-09-18T01:03:23.298 INFO:teuthology.orchestra.run.smithi164.stdout:2451 read 0x50c2800 thru 0x50c9190 (0x6991 bytes) 2024-09-18T01:03:23.299 INFO:teuthology.orchestra.run.smithi164.stdout:2452 write 0x7030000 thru 0x7037b8f (0x7b90 bytes) 2024-09-18T01:03:23.299 INFO:teuthology.orchestra.run.smithi164.stdout:2456 write 0x5814200 thru 0x581a4de (0x62df bytes) 2024-09-18T01:03:23.301 INFO:teuthology.orchestra.run.smithi164.stdout:2457 write 0xa6bf200 thru 0xa6cd806 (0xe607 bytes) 2024-09-18T01:03:23.302 INFO:teuthology.orchestra.run.smithi164.stdout:2461 read 0x7aed000 thru 0x7af3f84 (0x6f85 bytes) 2024-09-18T01:03:23.302 INFO:teuthology.orchestra.run.smithi164.stdout:2462 write 0x4989400 thru 0x4997b9f (0xe7a0 bytes) 2024-09-18T01:03:23.302 INFO:teuthology.orchestra.run.smithi164.stdout:2464 write 0x77b9e00 thru 0x77c209f (0x82a0 bytes) 2024-09-18T01:03:23.306 INFO:teuthology.orchestra.run.smithi164.stdout:2466 trunc from 0xc9a0c00 to 0xa4f0200 2024-09-18T01:03:23.332 INFO:teuthology.orchestra.run.smithi164.stdout:2467 punch from 0x640c200 to 0x6419a00, (0xd800 bytes) 2024-09-18T01:03:23.333 INFO:teuthology.orchestra.run.smithi164.stdout:2469 read 0xa059a00 thru 0xa066628 (0xcc29 bytes) 2024-09-18T01:03:23.334 INFO:teuthology.orchestra.run.smithi164.stdout:2471 read 0x65cd000 thru 0x65dcf9c (0xff9d bytes) 2024-09-18T01:03:23.335 INFO:teuthology.orchestra.run.smithi164.stdout:2472 read 0x3204e00 thru 0x3210aac (0xbcad bytes) 2024-09-18T01:03:23.342 INFO:teuthology.orchestra.run.smithi164.stdout:2473 trunc from 0xa4f0200 to 0x46eb000 2024-09-18T01:03:23.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:23.363 INFO:teuthology.orchestra.run.smithi164.stdout:2477 read 0x2da9c00 thru 0x2dadd62 (0x4163 bytes) 2024-09-18T01:03:23.365 INFO:teuthology.orchestra.run.smithi164.stdout:2480 read 0xcae00 thru 0xd359c (0x879d bytes) 2024-09-18T01:03:23.365 INFO:teuthology.orchestra.run.smithi164.stdout:2481 clone 16 order 23 su 1048576 sc 14 2024-09-18T01:03:23.366 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:23.367 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:23.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:23.417 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:23.416+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:23.443 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:03:23.492 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:23.490+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:23.495 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:23.496 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:23.515 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:23.525 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:23.525 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:23.525 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:23.532 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:23.532 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:23.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:23.550 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:23.569 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:23.569 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:23.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:23.587 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:23.592 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:23.591+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:23.662 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:23.662 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:23.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:23.715 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:23.714+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2015 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4265984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.463506944999999, 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027955569999999999, 2024-09-18T01:03:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2015 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:23.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318446+0000", 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318453+0000", 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318458+0000", 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9910000000000002e-06 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346240+0000", 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027782305 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346316+0000", 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5731999999999999e-05 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346399+0000", 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3137e-05 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.463486045, 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:23.789 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:23.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:23.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.463465536000001, 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:23.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.463445731, 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:23.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:23.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:23.804 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.463411091000001, 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:23.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.463389589999998, 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:23.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.463366412999999, 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:23.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:23.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2237 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4202496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459397628000001, 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028056793, 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2237 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322555+0000", 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7740000000000001e-06 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322562+0000", 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.426e-06 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322567+0000", 2024-09-18T01:03:23.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350490+0000", 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027923436999999999 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350528+0000", 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7660999999999997e-05 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350610+0000", 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2150000000000005e-05 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459377580999998, 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:23.824 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:23.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:23.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:23.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459358028, 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:23.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:23.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459337523999999, 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:23.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459316659999999, 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:23.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459296760000001, 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:23.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:23.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459272672000001, 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:23.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.853 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459241351999999, 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:23.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:23.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459196746, 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:23.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:23.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.863 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459165955, 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:23.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.45914247, 2024-09-18T01:03:23.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:23.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:23.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459113415000001, 2024-09-18T01:03:23.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:23.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.459092655999999, 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:23.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:23.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:23.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:23.883 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:23.884 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-18T01:03:23.884 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:23.883+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:23.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:23.985 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:23.984+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:24.044 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:24.045 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:24.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:24.085 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:24.085+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:24.172 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:24.172 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:24.172 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:24.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:24.186 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:24.185+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:24.287 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:24.286+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:24.293 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:24.293 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.014200596999999, 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:24.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:24.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:24.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:24.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.009574694000001, 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:24.304 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:24.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.009496644999999, 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:24.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:24.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:24.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1292 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1774592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:24.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.952724920000001, 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050913359999999998, 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1292 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338601+0000", 2024-09-18T01:03:24.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.024e-06 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338610+0000", 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5469999999999997e-06 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338616+0000", 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389393+0000", 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05077661 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389472+0000", 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9776000000000003e-05 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389512+0000", 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0000000000000003e-05 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.952700602, 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:24.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:24.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.952674737999999, 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:24.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.952652258000001, 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:24.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:24.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.952631222000001, 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:24.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.952603828000001, 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:24.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.952579523000001, 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:24.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:24.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.949180040000002, 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.949155480000002, 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.949113962999999, 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.949093461, 2024-09-18T01:03:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.949072995000002, 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.949031654999999, 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.949011617, 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.948969286000001, 2024-09-18T01:03:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.948948294000001, 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 25.948897324000001, 2024-09-18T01:03:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.421 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:24.422 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:24.421+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:24.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:24.423 INFO:teuthology.orchestra.run.smithi164.stdout:2482 clone 17 order 22 su 2097152 sc 12 2024-09-18T01:03:24.523 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:24.522+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:24.539 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:24.540 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:24.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:24.558 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone16 from 0x46eb000 (overlap 0x46eb000) to 0x0 2024-09-18T01:03:24.624 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:24.623+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:24.685 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:24.685 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:24.686 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:24.686 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:24.686 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:24.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:24.740 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:24.739+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.070208176000001, 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:24.807 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:24.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:24.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.070188217999998, 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:24.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:24.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:24.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.070169277000002, 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:24.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:24.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:24.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.070147767999998, 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.070122040000001, 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:24.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:24.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:24.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:24.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:24.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:24.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061390181, 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061367467, 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:24.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:24.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:24.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:24.853 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061348336999998, 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:24.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061329490999999, 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:24.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:24.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061310613, 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:24.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.872 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061288747999999, 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:24.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061265113000001, 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:24.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061231462999999, 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:24.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:24.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061203608, 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:24.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061183785000001, 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:24.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:24.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:24.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061163351000001, 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:24.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061141271, 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061118949000001, 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061096552999999, 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.061076134, 2024-09-18T01:03:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:24.938 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:24.939 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:24.940 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:24.938+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:24.956 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-18T01:03:25.040 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:25.039+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:25.063 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:25.063 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:25.064 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:25.064 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:25.074 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:25.141 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:25.140+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:25.187 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:25.187 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:25.187 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:25.188 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:25.188 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:25.197 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:25.243 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:25.242+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.26027878, 2024-09-18T01:03:25.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.308 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:03:25.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.311 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.260267629000001, 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.312 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.313 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:03:25.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:03:25.316 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.260255798999999, 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.317 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:25.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:03:25.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.260239116000001, 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:03:25.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:03:25.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.326 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.260227878999999, 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.327 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.328 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:03:25.329 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:03:25.331 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.258905773999999, 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.332 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.333 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.333 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.333 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:03:25.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.336 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:03:25.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.258893058999998, 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:03:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.345 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.258848075, 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:03:25.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:25.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:03:25.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:25.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:03:25.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:25.351 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.258820450999998, 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:03:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:03:25.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:03:25.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:25.357 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.255273072000001, 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:03:25.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.254912964999999, 2024-09-18T01:03:25.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:25.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:03:25.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.254886851999998, 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.367 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:25.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:03:25.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23940 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894162912] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.252717139000001, 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037903539999999999, 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.372 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23940 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051180+0000", 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4190000000000001e-06 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051183+0000", 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.774e-06 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051185+0000", 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5330000000000001e-06 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.088984+0000", 2024-09-18T01:03:25.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037798867999999999 2024-09-18T01:03:25.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089016+0000", 2024-09-18T01:03:25.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2333999999999999e-05 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089082+0000", 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5399000000000002e-05 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.252707464, 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:03:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.252697669, 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:25.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:03:25.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.25268689, 2024-09-18T01:03:25.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.252667094, 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.392 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:03:25.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.396 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.250883573999999, 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.397 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:03:25.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.401 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.250856735999999, 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:03:25.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 23.250846188000001, 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:25.406 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:25.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:03:25.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:25.410 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:25.411 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:25.412 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:25.411+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:25.413 INFO:teuthology.orchestra.run.smithi164.stdout:2483 write 0x9fe5e00 thru 0x9ff1eba (0xc0bb bytes) 2024-09-18T01:03:25.413 INFO:teuthology.orchestra.run.smithi164.stdout:2484 read 0x3c6c200 thru 0x3c7337e (0x717f bytes) 2024-09-18T01:03:25.413 INFO:teuthology.orchestra.run.smithi164.stdout:2486 write 0x24c8a00 thru 0x24d5f3b (0xd53c bytes) 2024-09-18T01:03:25.413 INFO:teuthology.orchestra.run.smithi164.stdout:2487 write 0x48f7a00 thru 0x48fed46 (0x7347 bytes) 2024-09-18T01:03:25.413 INFO:teuthology.orchestra.run.smithi164.stdout:2488 trunc from 0x9ff2000 to 0x48d2a00 2024-09-18T01:03:25.413 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:25.428 INFO:teuthology.orchestra.run.smithi164.stdout:2491 read 0x219a800 thru 0x21a8d6d (0xe56e bytes) 2024-09-18T01:03:25.446 INFO:teuthology.orchestra.run.smithi164.stdout:2492 write 0xaa62600 thru 0xaa7165d (0xf05e bytes) 2024-09-18T01:03:25.446 INFO:teuthology.orchestra.run.smithi164.stdout:2493 punch from 0x9b3de00 to 0x9b47600, (0x9800 bytes) 2024-09-18T01:03:25.450 INFO:teuthology.orchestra.run.smithi164.stdout:2494 trunc from 0xaa71800 to 0xd820a00 2024-09-18T01:03:25.464 INFO:teuthology.orchestra.run.smithi164.stdout:2498 read 0x63b0a00 thru 0x63ba0fc (0x96fd bytes) 2024-09-18T01:03:25.464 INFO:teuthology.orchestra.run.smithi164.stdout:2499 read 0x3f9b400 thru 0x3fa004a (0x4c4b bytes) 2024-09-18T01:03:25.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:03:25.465 INFO:teuthology.orchestra.run.smithi164.stdout:2502 write 0x95f9200 thru 0x95f9b7d (0x97e bytes) 2024-09-18T01:03:25.475 INFO:teuthology.orchestra.run.smithi164.stdout:2503 trunc from 0xd820a00 to 0x46c3e00 2024-09-18T01:03:25.513 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:25.512+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:25.558 INFO:teuthology.orchestra.run.smithi164.stdout:2504 read 0x2f2fc00 thru 0x2f39b20 (0x9f21 bytes) 2024-09-18T01:03:25.559 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:25.559 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:25.559 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:25.559 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:25.569 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:25.582 INFO:teuthology.orchestra.run.smithi164.stdout:2505 write 0xcb54200 thru 0xcb59c7f (0x5a80 bytes) 2024-09-18T01:03:25.589 INFO:teuthology.orchestra.run.smithi164.stdout:2506 write 0xb37cc00 thru 0xb389acd (0xcece bytes) 2024-09-18T01:03:25.589 INFO:teuthology.orchestra.run.smithi164.stdout:2508 write 0x4444c00 thru 0x4445780 (0xb81 bytes) 2024-09-18T01:03:25.592 INFO:teuthology.orchestra.run.smithi164.stdout:2509 trunc from 0xcb59e00 to 0xe92d200 2024-09-18T01:03:25.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:25.608 INFO:teuthology.orchestra.run.smithi164.stdout:2511 write 0xc269800 thru 0xc26b1fc (0x19fd bytes) 2024-09-18T01:03:25.610 INFO:teuthology.orchestra.run.smithi164.stdout:2512 trunc from 0xe92d200 to 0xd23f000 2024-09-18T01:03:25.613 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:25.612+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:25.627 INFO:teuthology.orchestra.run.smithi164.stdout:2513 punch from 0x5ede600 to 0x5eeae00, (0xc800 bytes) 2024-09-18T01:03:25.628 INFO:teuthology.orchestra.run.smithi164.stdout:2514 read 0xcb9e200 thru 0xcba1f2c (0x3d2d bytes) 2024-09-18T01:03:25.629 INFO:teuthology.orchestra.run.smithi164.stdout:2515 read 0x3e6bc00 thru 0x3e6e9d4 (0x2dd5 bytes) 2024-09-18T01:03:25.629 INFO:teuthology.orchestra.run.smithi164.stdout:2520 read 0x6c18400 thru 0x6c20109 (0x7d0a bytes) 2024-09-18T01:03:25.629 INFO:teuthology.orchestra.run.smithi164.stdout:2521 write 0x4f05a00 thru 0x4f14011 (0xe612 bytes) 2024-09-18T01:03:25.630 INFO:teuthology.orchestra.run.smithi164.stdout:2522 write 0x1634000 thru 0x1643544 (0xf545 bytes) 2024-09-18T01:03:25.637 INFO:teuthology.orchestra.run.smithi164.stdout:2524 trunc from 0xd23f000 to 0x78dc800 2024-09-18T01:03:25.660 INFO:teuthology.orchestra.run.smithi164.stdout:2526 punch from 0x2a84200 to 0x2a92800, (0xe600 bytes) 2024-09-18T01:03:25.662 INFO:teuthology.orchestra.run.smithi164.stdout:2527 read 0x5dcbc00 thru 0x5dcf845 (0x3c46 bytes) 2024-09-18T01:03:25.663 INFO:teuthology.orchestra.run.smithi164.stdout:2529 read 0x11ca800 thru 0x11d99bb (0xf1bc bytes) 2024-09-18T01:03:25.663 INFO:teuthology.orchestra.run.smithi164.stdout:2530 read 0x457ec00 thru 0x458b670 (0xca71 bytes) 2024-09-18T01:03:25.664 INFO:teuthology.orchestra.run.smithi164.stdout:2531 write 0x5f1ac00 thru 0x5f2259b (0x799c bytes) 2024-09-18T01:03:25.665 INFO:teuthology.orchestra.run.smithi164.stdout:2533 punch from 0x71c200 to 0x724200, (0x8000 bytes) 2024-09-18T01:03:25.665 INFO:teuthology.orchestra.run.smithi164.stdout:2534 read 0x3297a00 thru 0x329e348 (0x6949 bytes) 2024-09-18T01:03:25.681 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:25.681 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:25.682 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:25.682 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:25.682 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:25.688 INFO:teuthology.orchestra.run.smithi164.stdout:2535 write 0xc9d3600 thru 0xc9d9033 (0x5a34 bytes) 2024-09-18T01:03:25.689 INFO:teuthology.orchestra.run.smithi164.stdout:2536 punch from 0x6826e00 to 0x682f800, (0x8a00 bytes) 2024-09-18T01:03:25.691 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:25.691 INFO:teuthology.orchestra.run.smithi164.stdout:2537 trunc from 0xc9d9200 to 0xb191800 2024-09-18T01:03:25.711 INFO:teuthology.orchestra.run.smithi164.stdout:2539 read 0x334d800 thru 0x3359d38 (0xc539 bytes) 2024-09-18T01:03:25.715 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:25.713+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:25.715 INFO:teuthology.orchestra.run.smithi164.stdout:2540 trunc from 0xb191800 to 0x8fc7200 2024-09-18T01:03:25.727 INFO:teuthology.orchestra.run.smithi164.stdout:2541 trunc from 0x8fc7200 to 0x8c18400 2024-09-18T01:03:25.739 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:25.749 INFO:teuthology.orchestra.run.smithi164.stdout:2542 read 0x55e5200 thru 0x55eb508 (0x6309 bytes) 2024-09-18T01:03:25.749 INFO:teuthology.orchestra.run.smithi164.stdout:2544 write 0x864be00 thru 0x864f6a3 (0x38a4 bytes) 2024-09-18T01:03:25.749 INFO:teuthology.orchestra.run.smithi164.stdout:2546 read 0x484a800 thru 0x484b030 (0x831 bytes) 2024-09-18T01:03:25.750 INFO:teuthology.orchestra.run.smithi164.stdout:2547 read 0x7c12800 thru 0x7c17d74 (0x5575 bytes) 2024-09-18T01:03:25.750 INFO:teuthology.orchestra.run.smithi164.stdout:2548 read 0x5ab4600 thru 0x5abe2e5 (0x9ce6 bytes) 2024-09-18T01:03:25.750 INFO:teuthology.orchestra.run.smithi164.stdout:2549 punch from 0x1e68a00 to 0x1e68e00, (0x400 bytes) 2024-09-18T01:03:25.761 INFO:teuthology.orchestra.run.smithi164.stdout:2551 write 0xa1b3200 thru 0xa1b7e68 (0x4c69 bytes) 2024-09-18T01:03:25.761 INFO:teuthology.orchestra.run.smithi164.stdout:2552 punch from 0x4690600 to 0x4696200, (0x5c00 bytes) 2024-09-18T01:03:25.764 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:25.765 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:25.765 INFO:teuthology.orchestra.run.smithi164.stdout:2553 read 0x3f1ca00 thru 0x3f21970 (0x4f71 bytes) 2024-09-18T01:03:25.766 INFO:teuthology.orchestra.run.smithi164.stdout:2555 write 0x3d85600 thru 0x3d8ed72 (0x9773 bytes) 2024-09-18T01:03:25.767 INFO:teuthology.orchestra.run.smithi164.stdout:2556 write 0x3c2c600 thru 0x3c3891c (0xc31d bytes) 2024-09-18T01:03:25.767 INFO:teuthology.orchestra.run.smithi164.stdout:2559 read 0xa08e200 thru 0xa09ced6 (0xecd7 bytes) 2024-09-18T01:03:25.768 INFO:teuthology.orchestra.run.smithi164.stdout:2560 write 0x18caa00 thru 0x18ce4f7 (0x3af8 bytes) 2024-09-18T01:03:25.768 INFO:teuthology.orchestra.run.smithi164.stdout:2561 write 0x4b1e000 thru 0x4b29cb6 (0xbcb7 bytes) 2024-09-18T01:03:25.769 INFO:teuthology.orchestra.run.smithi164.stdout:2562 read 0xa012600 thru 0xa019dba (0x77bb bytes) 2024-09-18T01:03:25.769 INFO:teuthology.orchestra.run.smithi164.stdout:2563 read 0x566be00 thru 0x567b8e7 (0xfae8 bytes) 2024-09-18T01:03:25.775 INFO:teuthology.orchestra.run.smithi164.stdout:2564 trunc from 0xa1b8000 to 0x5620200 2024-09-18T01:03:25.784 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:25.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:25.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:25.815 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:25.815+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:25.819 INFO:teuthology.orchestra.run.smithi164.stdout:2567 write 0xb40600 thru 0xb505eb (0xffec bytes) 2024-09-18T01:03:25.821 INFO:teuthology.orchestra.run.smithi164.stdout:2569 punch from 0x2c08200 to 0x2c14e00, (0xcc00 bytes) 2024-09-18T01:03:25.821 INFO:teuthology.orchestra.run.smithi164.stdout:2570 trunc from 0x5620200 to 0x560ec00 2024-09-18T01:03:25.825 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:25.826 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:25.845 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:25.845 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:25.847 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:25.847 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:25.848 INFO:teuthology.orchestra.run.smithi164.stdout:2573 punch from 0x38c4c00 to 0x38c8800, (0x3c00 bytes) 2024-09-18T01:03:25.848 INFO:teuthology.orchestra.run.smithi164.stdout:2574 write 0x1ec9200 thru 0x1ed6b4a (0xd94b bytes) 2024-09-18T01:03:25.849 INFO:teuthology.orchestra.run.smithi164.stdout:2576 read 0x25dce00 thru 0x25de0f1 (0x12f2 bytes) 2024-09-18T01:03:25.850 INFO:teuthology.orchestra.run.smithi164.stdout:2577 read 0x1bdb400 thru 0x1be9d33 (0xe934 bytes) 2024-09-18T01:03:25.855 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:25.859 INFO:teuthology.orchestra.run.smithi164.stdout:2578 trunc from 0x560ec00 to 0xc223400 2024-09-18T01:03:25.880 INFO:teuthology.orchestra.run.smithi164.stdout:2580 read 0x4d9ba00 thru 0x4d9c438 (0xa39 bytes) 2024-09-18T01:03:25.881 INFO:teuthology.orchestra.run.smithi164.stdout:2583 read 0x4320400 thru 0x432dcd3 (0xd8d4 bytes) 2024-09-18T01:03:25.883 INFO:teuthology.orchestra.run.smithi164.stdout:2584 trunc from 0xc223400 to 0xa5bc200 2024-09-18T01:03:25.902 INFO:teuthology.orchestra.run.smithi164.stdout:2587 read 0x3037c00 thru 0x3040864 (0x8c65 bytes) 2024-09-18T01:03:25.904 INFO:teuthology.orchestra.run.smithi164.stdout:2589 trunc from 0xa5bc200 to 0x91fe600 2024-09-18T01:03:25.916 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:25.915+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:25.921 INFO:teuthology.orchestra.run.smithi164.stdout:2591 read 0x6254400 thru 0x6257891 (0x3492 bytes) 2024-09-18T01:03:25.921 INFO:teuthology.orchestra.run.smithi164.stdout:2592 read 0x829de00 thru 0x82a7a6f (0x9c70 bytes) 2024-09-18T01:03:25.945 INFO:teuthology.orchestra.run.smithi164.stdout:2597 write 0xc96ce00 thru 0xc978fa4 (0xc1a5 bytes) 2024-09-18T01:03:25.958 INFO:teuthology.orchestra.run.smithi164.stdout:2598 trunc from 0xc979000 to 0x2411a00 2024-09-18T01:03:25.960 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:25.960 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:25.969 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:25.991 INFO:teuthology.orchestra.run.smithi164.stdout:2600 punch from 0xfad200 to 0xfbbc00, (0xea00 bytes) 2024-09-18T01:03:26.005 INFO:teuthology.orchestra.run.smithi164.stdout:2602 trunc from 0x2411a00 to 0xe5f1a00 2024-09-18T01:03:26.014 INFO:teuthology.orchestra.run.smithi164.stdout:2603 read 0x538be00 thru 0x538eab1 (0x2cb2 bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2605 write 0x3c1a000 thru 0x3c1c245 (0x2246 bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2606 write 0x52bdc00 thru 0x52be63c (0xa3d bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2609 read 0x7d5ca00 thru 0x7d61554 (0x4b55 bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2610 read 0x86c3e00 thru 0x86cac48 (0x6e49 bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2613 read 0x3630200 thru 0x36344e6 (0x42e7 bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2614 read 0x3e15600 thru 0x3e23a9c (0xe49d bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2616 write 0x869f200 thru 0x86a1f5f (0x2d60 bytes) 2024-09-18T01:03:26.016 INFO:teuthology.orchestra.run.smithi164.stdout:2618 write 0xa14a00 thru 0xa20552 (0xbb53 bytes) 2024-09-18T01:03:26.017 INFO:teuthology.orchestra.run.smithi164.stdout:2619 read 0x5b99400 thru 0x5ba5fea (0xcbeb bytes) 2024-09-18T01:03:26.017 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:26.016+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:26.019 INFO:teuthology.orchestra.run.smithi164.stdout:2621 trunc from 0xe5f1a00 to 0xc592800 2024-09-18T01:03:26.033 INFO:teuthology.orchestra.run.smithi164.stdout:2622 write 0x94c2c00 thru 0x94cbbc5 (0x8fc6 bytes) 2024-09-18T01:03:26.033 INFO:teuthology.orchestra.run.smithi164.stdout:2623 punch from 0x61b2800 to 0x61bba00, (0x9200 bytes) 2024-09-18T01:03:26.034 INFO:teuthology.orchestra.run.smithi164.stdout:2624 read 0x5f49200 thru 0x5f50715 (0x7516 bytes) 2024-09-18T01:03:26.034 INFO:teuthology.orchestra.run.smithi164.stdout:2626 read 0x4156e00 thru 0x415cb6f (0x5d70 bytes) 2024-09-18T01:03:26.034 INFO:teuthology.orchestra.run.smithi164.stdout:2629 punch from 0x9d15200 to 0x9d23c00, (0xea00 bytes) 2024-09-18T01:03:26.035 INFO:teuthology.orchestra.run.smithi164.stdout:2630 read 0xabb1800 thru 0xabb72fd (0x5afe bytes) 2024-09-18T01:03:26.035 INFO:teuthology.orchestra.run.smithi164.stdout:2631 read 0x3e2e600 thru 0x3e3450a (0x5f0b bytes) 2024-09-18T01:03:26.035 INFO:teuthology.orchestra.run.smithi164.stdout:2632 clone 18 order 21 su 262144 sc 5 2024-09-18T01:03:26.112 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:26.112 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:26.118 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:26.117+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:26.120 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:26.219 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:26.218+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:26.263 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:26.263 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:26.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:26.320 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:26.319+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:26.389 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:26.389 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:26.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:26.421 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:26.420+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:26.455 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:03:26.512 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:26.512 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:26.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:26.523 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:26.521+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:26.575 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone17 intact 2024-09-18T01:03:26.623 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:26.621+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:26.671 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:26.671 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:26.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:26.724 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:26.723+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:26.791 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:26.791 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:26.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:26.825 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:26.824+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:26.926 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:26.925+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:26.948 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:26.949 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:26.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:26.976 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-18T01:03:27.032 INFO:teuthology.orchestra.run.smithi164.stdout:2635 write 0x485e400 thru 0x486ac08 (0xc809 bytes) 2024-09-18T01:03:27.042 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:27.041+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:27.092 INFO:teuthology.orchestra.run.smithi164.stdout:2636 read 0x41b0400 thru 0x41bd349 (0xcf4a bytes) 2024-09-18T01:03:27.093 INFO:teuthology.orchestra.run.smithi164.stdout:2639 read 0x773200 thru 0x779695 (0x6496 bytes) 2024-09-18T01:03:27.110 INFO:teuthology.orchestra.run.smithi164.stdout:2640 write 0xcec0c00 thru 0xcec18e1 (0xce2 bytes) 2024-09-18T01:03:27.111 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:27.111 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:27.112 INFO:teuthology.orchestra.run.smithi164.stdout:2641 trunc from 0xcec1a00 to 0xd8ce000 2024-09-18T01:03:27.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:27.127 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:27.126+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:27.137 INFO:teuthology.orchestra.run.smithi164.stdout:2645 trunc from 0xd8ce000 to 0x28bb800 2024-09-18T01:03:27.182 INFO:teuthology.orchestra.run.smithi164.stdout:2648 write 0xe39e600 thru 0xe3a3ffb (0x59fc bytes) 2024-09-18T01:03:27.183 INFO:teuthology.orchestra.run.smithi164.stdout:2649 punch from 0x65b5a00 to 0x65c0600, (0xac00 bytes) 2024-09-18T01:03:27.184 INFO:teuthology.orchestra.run.smithi164.stdout:2651 write 0x2b01000 thru 0x2b060c7 (0x50c8 bytes) 2024-09-18T01:03:27.184 INFO:teuthology.orchestra.run.smithi164.stdout:2652 read 0x37f4a00 thru 0x37f719b (0x279c bytes) 2024-09-18T01:03:27.185 INFO:teuthology.orchestra.run.smithi164.stdout:2653 read 0x31dac00 thru 0x31dc1a4 (0x15a5 bytes) 2024-09-18T01:03:27.185 INFO:teuthology.orchestra.run.smithi164.stdout:2656 write 0xd461800 thru 0xd4647e6 (0x2fe7 bytes) 2024-09-18T01:03:27.185 INFO:teuthology.orchestra.run.smithi164.stdout:2660 write 0x30b9400 thru 0x30bf28e (0x5e8f bytes) 2024-09-18T01:03:27.185 INFO:teuthology.orchestra.run.smithi164.stdout:2661 read 0x1609000 thru 0x161293d (0x993e bytes) 2024-09-18T01:03:27.185 INFO:teuthology.orchestra.run.smithi164.stdout:2662 punch from 0x2b9a00 to 0x2bd200, (0x3800 bytes) 2024-09-18T01:03:27.186 INFO:teuthology.orchestra.run.smithi164.stdout:2664 write 0x63fce00 thru 0x64028bf (0x5ac0 bytes) 2024-09-18T01:03:27.186 INFO:teuthology.orchestra.run.smithi164.stdout:2665 read 0xaf59a00 thru 0xaf69566 (0xfb67 bytes) 2024-09-18T01:03:27.201 INFO:teuthology.orchestra.run.smithi164.stdout:2666 trunc from 0xe3a4000 to 0x16de000 2024-09-18T01:03:27.228 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:27.227+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:27.241 INFO:teuthology.orchestra.run.smithi164.stdout:2667 read 0xd15e00 thru 0xd22f78 (0xd179 bytes) 2024-09-18T01:03:27.261 INFO:teuthology.orchestra.run.smithi164.stdout:2668 write 0x311ce00 thru 0x311f173 (0x2374 bytes) 2024-09-18T01:03:27.284 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:27.284 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:27.293 INFO:teuthology.orchestra.run.smithi164.stdout:2670 write 0xa613c00 thru 0xa61f7b6 (0xbbb7 bytes) 2024-09-18T01:03:27.293 INFO:teuthology.orchestra.run.smithi164.stdout:2671 write 0x5ecbc00 thru 0x5eda5bb (0xe9bc bytes) 2024-09-18T01:03:27.293 INFO:teuthology.orchestra.run.smithi164.stdout:2672 read 0x6be1c00 thru 0x6bed0f5 (0xb4f6 bytes) 2024-09-18T01:03:27.293 INFO:teuthology.orchestra.run.smithi164.stdout:2674 read 0x3086000 thru 0x30864b2 (0x4b3 bytes) 2024-09-18T01:03:27.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:27.318 INFO:teuthology.orchestra.run.smithi164.stdout:2675 write 0xe548000 thru 0xe54adc3 (0x2dc4 bytes) 2024-09-18T01:03:27.318 INFO:teuthology.orchestra.run.smithi164.stdout:2676 punch from 0xe115200 to 0xe117e00, (0x2c00 bytes) 2024-09-18T01:03:27.318 INFO:teuthology.orchestra.run.smithi164.stdout:2677 write 0x95f4000 thru 0x9600fd9 (0xcfda bytes) 2024-09-18T01:03:27.318 INFO:teuthology.orchestra.run.smithi164.stdout:2682 write 0xb6a6e00 thru 0xb6a88c8 (0x1ac9 bytes) 2024-09-18T01:03:27.318 INFO:teuthology.orchestra.run.smithi164.stdout:2684 write 0xc4c000 thru 0xc51082 (0x5083 bytes) 2024-09-18T01:03:27.319 INFO:teuthology.orchestra.run.smithi164.stdout:2685 write 0xc2ff000 thru 0xc305b06 (0x6b07 bytes) 2024-09-18T01:03:27.319 INFO:teuthology.orchestra.run.smithi164.stdout:2686 read 0x77af200 thru 0x77b1721 (0x2522 bytes) 2024-09-18T01:03:27.319 INFO:teuthology.orchestra.run.smithi164.stdout:2688 write 0xb104000 thru 0xb112762 (0xe763 bytes) 2024-09-18T01:03:27.319 INFO:teuthology.orchestra.run.smithi164.stdout:2689 write 0xb767200 thru 0xb76e97f (0x7780 bytes) 2024-09-18T01:03:27.320 INFO:teuthology.orchestra.run.smithi164.stdout:2690 write 0x8b84800 thru 0x8b8880d (0x400e bytes) 2024-09-18T01:03:27.320 INFO:teuthology.orchestra.run.smithi164.stdout:2691 read 0x9820a00 thru 0x982a754 (0x9d55 bytes) 2024-09-18T01:03:27.321 INFO:teuthology.orchestra.run.smithi164.stdout:2692 trunc from 0xe54ae00 to 0xd160a00 2024-09-18T01:03:27.329 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:27.328+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:27.347 INFO:teuthology.orchestra.run.smithi164.stdout:2696 read 0x9eb1a00 thru 0x9ebefc9 (0xd5ca bytes) 2024-09-18T01:03:27.360 INFO:teuthology.orchestra.run.smithi164.stdout:2697 trunc from 0xd160a00 to 0x32b5400 2024-09-18T01:03:27.375 INFO:teuthology.orchestra.run.smithi164.stdout:2698 punch from 0x12f8e00 to 0x12ff400, (0x6600 bytes) 2024-09-18T01:03:27.377 INFO:teuthology.orchestra.run.smithi164.stdout:2701 punch from 0x163e00 to 0x16d600, (0x9800 bytes) 2024-09-18T01:03:27.377 INFO:teuthology.orchestra.run.smithi164.stdout:2702 punch from 0x7c1200 to 0x7cda00, (0xc800 bytes) 2024-09-18T01:03:27.377 INFO:teuthology.orchestra.run.smithi164.stdout:2704 clone 19 order 19 su 8192 sc 14 2024-09-18T01:03:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:27.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:27.430 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:27.429+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:27.530 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:27.529+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:27.541 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone18 from 0x32b5400 (overlap 0x16de000) to 0x249600 2024-09-18T01:03:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:27.557 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:27.631 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:27.630+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:27.645 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:27.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:03:27.664 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:27.664 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:27.673 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:27.731 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:27.731+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:27.821 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:27.821 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:27.830 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:27.832 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:27.831+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:27.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:27.915 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-18T01:03:27.933 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:27.932+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:27.936 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:27.936 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:27.944 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:28.009 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:28.029 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:28.030 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:28.034 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:28.033+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:28.049 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:28.067 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:28.067 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:28.083 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:28.084 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:28.095 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:28.096 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:28.096 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:28.096 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:28.105 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:28.134 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:28.133+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:28.235 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:28.234+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:28.253 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:28.253 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:28.253 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:28.253 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:28.253 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:28.263 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:28.347 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:28.346+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:28.409 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.111348059000001, 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:03:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:03:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.111323941999999, 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:03:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.111244702, 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:03:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.111208482999999, 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.111179870000001, 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:03:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:03:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.111159005000001, 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.110541288, 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:03:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.110512021000002, 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.110445729999999, 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.110324963, 2024-09-18T01:03:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:03:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.110106516999998, 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.110085156, 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.458 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109931217, 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:03:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109911498999999, 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:03:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109891188999999, 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:03:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109871302999998, 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:03:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109851343999999, 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:03:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109831346, 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:03:28.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:03:28.488 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109811679, 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:03:28.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:28.493 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 22.109791777000002, 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:03:28.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.497 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.498 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.498 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:28.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:03:28.500 INFO:teuthology.orchestra.run.smithi164.stdout:2706 punch from 0x2c81000 to 0x2c89000, (0x8000 bytes) 2024-09-18T01:03:28.500 INFO:teuthology.orchestra.run.smithi164.stdout:2709 read 0x2a74c00 thru 0x2a7ef3d (0xa33e bytes) 2024-09-18T01:03:28.500 INFO:teuthology.orchestra.run.smithi164.stdout:2710 punch from 0xd1e600 to 0xd28200, (0x9c00 bytes) 2024-09-18T01:03:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:03:28.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:28.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:03:28.502 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:28.498+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:28.515 INFO:teuthology.orchestra.run.smithi164.stdout:2713 read 0x2249000 thru 0x224c1c9 (0x31ca bytes) 2024-09-18T01:03:28.528 INFO:teuthology.orchestra.run.smithi164.stdout:2714 trunc from 0x32b5400 to 0xdd1ae00 2024-09-18T01:03:28.537 INFO:teuthology.orchestra.run.smithi164.stdout:2717 read 0x7a73e00 thru 0x7a7e6ff (0xa900 bytes) 2024-09-18T01:03:28.538 INFO:teuthology.orchestra.run.smithi164.stdout:2718 write 0x7129000 thru 0x712de1a (0x4e1b bytes) 2024-09-18T01:03:28.538 INFO:teuthology.orchestra.run.smithi164.stdout:2724 write 0x905d000 thru 0x906abe3 (0xdbe4 bytes) 2024-09-18T01:03:28.538 INFO:teuthology.orchestra.run.smithi164.stdout:2725 read 0x69e1200 thru 0x69ed5ae (0xc3af bytes) 2024-09-18T01:03:28.538 INFO:teuthology.orchestra.run.smithi164.stdout:2727 read 0x97cd200 thru 0x97d9400 (0xc201 bytes) 2024-09-18T01:03:28.538 INFO:teuthology.orchestra.run.smithi164.stdout:2729 read 0x1a96400 thru 0x1aa55e6 (0xf1e7 bytes) 2024-09-18T01:03:28.551 INFO:teuthology.orchestra.run.smithi164.stdout:2730 trunc from 0xdd1ae00 to 0x3ab8a00 2024-09-18T01:03:28.585 INFO:teuthology.orchestra.run.smithi164.stdout:2732 trunc from 0x3ab8a00 to 0x5fc2a00 2024-09-18T01:03:28.599 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:28.598+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout:2733 write 0xc6f7400 thru 0xc6f7669 (0x26a bytes) 2024-09-18T01:03:28.620 INFO:teuthology.orchestra.run.smithi164.stdout:2734 punch from 0x97a1e00 to 0x97a6e00, (0x5000 bytes) 2024-09-18T01:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout:2738 write 0x714ca00 thru 0x71574f9 (0xaafa bytes) 2024-09-18T01:03:28.621 INFO:teuthology.orchestra.run.smithi164.stdout:2739 punch from 0x5513c00 to 0x551f600, (0xba00 bytes) 2024-09-18T01:03:28.635 INFO:teuthology.orchestra.run.smithi164.stdout:2741 trunc from 0xc6f7800 to 0x74e800 2024-09-18T01:03:28.667 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:28.668 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:28.668 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:28.668 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:28.677 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:28.701 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:28.700+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:28.703 INFO:teuthology.orchestra.run.smithi164.stdout:2744 punch from 0x5d1800 to 0x5d4a00, (0x3200 bytes) 2024-09-18T01:03:28.730 INFO:teuthology.orchestra.run.smithi164.stdout:2745 write 0x899ce00 thru 0x89ac597 (0xf798 bytes) 2024-09-18T01:03:28.740 INFO:teuthology.orchestra.run.smithi164.stdout:2746 trunc from 0x89ac600 to 0x11e00 2024-09-18T01:03:28.802 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:28.801+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:28.804 INFO:teuthology.orchestra.run.smithi164.stdout:2749 read 0x5e00 thru 0x73b6 (0x15b7 bytes) 2024-09-18T01:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout:2750 punch from 0xea00 to 0x10e00, (0x2400 bytes) 2024-09-18T01:03:28.812 INFO:teuthology.orchestra.run.smithi164.stdout:2751 punch from 0x1a00 to 0x9e00, (0x8400 bytes) 2024-09-18T01:03:28.832 INFO:teuthology.orchestra.run.smithi164.stdout:2752 write 0xb082a00 thru 0xb0840c9 (0x16ca bytes) 2024-09-18T01:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout:2753 write 0x8e03800 thru 0x8e0c0e5 (0x88e6 bytes) 2024-09-18T01:03:28.833 INFO:teuthology.orchestra.run.smithi164.stdout:2755 punch from 0x103fe00 to 0x1049600, (0x9800 bytes) 2024-09-18T01:03:28.837 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:28.837 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:28.837 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:28.837 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:28.837 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:28.849 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:28.866 INFO:teuthology.orchestra.run.smithi164.stdout:2756 write 0xd15a000 thru 0xd15f5e8 (0x55e9 bytes) 2024-09-18T01:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:2757 read 0x60dc400 thru 0x60de1f8 (0x1df9 bytes) 2024-09-18T01:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:2759 punch from 0x6490200 to 0x6492a00, (0x2800 bytes) 2024-09-18T01:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:2761 read 0x202d000 thru 0x2039168 (0xc169 bytes) 2024-09-18T01:03:28.867 INFO:teuthology.orchestra.run.smithi164.stdout:2763 punch from 0xa5acc00 to 0xa5b1400, (0x4800 bytes) 2024-09-18T01:03:28.891 INFO:teuthology.orchestra.run.smithi164.stdout:2764 write 0xebfac00 thru 0xebfd94c (0x2d4d bytes) 2024-09-18T01:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout:2765 punch from 0x1f12400 to 0x1f1cc00, (0xa800 bytes) 2024-09-18T01:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout:2766 write 0xd029600 thru 0xd02d5d3 (0x3fd4 bytes) 2024-09-18T01:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout:2770 read 0x14d2000 thru 0x14e0625 (0xe626 bytes) 2024-09-18T01:03:28.892 INFO:teuthology.orchestra.run.smithi164.stdout:2771 clone 20 order 22 su 65536 sc 2 2024-09-18T01:03:28.903 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:28.901+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:28.962 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 56.586182116000003, 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.588170810999998, 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.580493859000001, 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.482635909000003, 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.626554896999998, 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.626185601000003, 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.624446755999998, 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.624374865999997, 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.536508855999998, 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.530317597, 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:29.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:29.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.528787911999999, 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:29.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:29.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.528408223, 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.032 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.512909819999997, 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:29.038 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:29.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:29.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.044 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.512703061000003, 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:29.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:29.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.050 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.512285304000002, 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.051 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:29.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.512076065999999, 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:29.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:29.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:29.060 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.512012132000002, 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:29.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:29.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.511263927000002, 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.065 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:29.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.069 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.252325122000002, 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:29.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.251461503999998, 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.076 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:29.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:29.081 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:29.082 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:29.083 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:29.082+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:29.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:29.184 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:29.183+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:29.191 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:29.191 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:29.191 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:29.191 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:29.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:29.284 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:29.283+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:29.308 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:29.308 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:29.308 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:29.308 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:29.308 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:29.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:29.386 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:29.385+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:29.467 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:29.467 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:29.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:29.467 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:29.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.467 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2015 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4265984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.146444346999999, 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027955569999999999, 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2015 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318446+0000", 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318453+0000", 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318458+0000", 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9910000000000002e-06 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346240+0000", 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027782305 2024-09-18T01:03:29.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346316+0000", 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5731999999999999e-05 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346399+0000", 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3137e-05 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.146423447, 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:29.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.146402938000001, 2024-09-18T01:03:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:29.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:29.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:29.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.146383133, 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.484 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.488 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.146348493000001, 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:29.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.146326991999999, 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:29.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.146303815, 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:29.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:29.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:29.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2237 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4202496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142335030000002, 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028056793, 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2237 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:29.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322555+0000", 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7740000000000001e-06 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322562+0000", 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.426e-06 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322567+0000", 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350490+0000", 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027923436999999999 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350528+0000", 2024-09-18T01:03:29.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7660999999999997e-05 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350610+0000", 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2150000000000005e-05 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142314982999999, 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:29.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142295430000001, 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:29.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:29.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142274925999999, 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142254061999999, 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:29.530 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142234162000001, 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:29.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:29.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142210074000001, 2024-09-18T01:03:29.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:29.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142178754, 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:29.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:29.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:29.546 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142134148, 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:29.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:29.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142103357, 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:29.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142079872, 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:29.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.560 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142050817000001, 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:29.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:29.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:29.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.142030058, 2024-09-18T01:03:29.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:29.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:29.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:29.572 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:29.573 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone19 from 0xebfda00 (overlap 0x11e00) to 0x5400 2024-09-18T01:03:29.573 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:03:29.574 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:29.572+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:29.574 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:03:29.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:29.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:03:29.674 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:29.673+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:29.729 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:29.729 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:29.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:29.780 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:29.778+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:29.876 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:29.875+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:29.895 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:29.895 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:29.896 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:29.896 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:29.896 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:29.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:29.977 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:29.975+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:30.004 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-18T01:03:30.017 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.738247479000002, 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.733621575999997, 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:30.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.733543527000002, 2024-09-18T01:03:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1292 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1774592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.676771802000001, 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050913359999999998, 2024-09-18T01:03:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1292 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338601+0000", 2024-09-18T01:03:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.024e-06 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338610+0000", 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5469999999999997e-06 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338616+0000", 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389393+0000", 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05077661 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389472+0000", 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9776000000000003e-05 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389512+0000", 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0000000000000003e-05 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.676747484, 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.676721619999999, 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.67669914, 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.676678104000001, 2024-09-18T01:03:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.676650710000001, 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.676626405, 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673226922000001, 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673202362000001, 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673160844999998, 2024-09-18T01:03:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673140343, 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673119877000001, 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673078536999999, 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673058499, 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.673016168, 2024-09-18T01:03:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.672995176000001, 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.672944206, 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:30.124 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:30.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:30.126 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:30.123+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:30.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:30.127 INFO:teuthology.orchestra.run.smithi164.stdout:2772 write 0x3559600 thru 0x355d78a (0x418b bytes) 2024-09-18T01:03:30.127 INFO:teuthology.orchestra.run.smithi164.stdout:2774 write 0x934ac00 thru 0x9353552 (0x8953 bytes) 2024-09-18T01:03:30.127 INFO:teuthology.orchestra.run.smithi164.stdout:2775 punch from 0x45bda00 to 0x45c8800, (0xae00 bytes) 2024-09-18T01:03:30.128 INFO:teuthology.orchestra.run.smithi164.stdout:2780 write 0xae04600 thru 0xae0ccd1 (0x86d2 bytes) 2024-09-18T01:03:30.128 INFO:teuthology.orchestra.run.smithi164.stdout:2782 read 0x403d800 thru 0x404cbf8 (0xf3f9 bytes) 2024-09-18T01:03:30.128 INFO:teuthology.orchestra.run.smithi164.stdout:2783 write 0xd471c00 thru 0xd47df61 (0xc362 bytes) 2024-09-18T01:03:30.128 INFO:teuthology.orchestra.run.smithi164.stdout:2785 read 0x7953200 thru 0x795a6f8 (0x74f9 bytes) 2024-09-18T01:03:30.128 INFO:teuthology.orchestra.run.smithi164.stdout:2787 trunc from 0xebfda00 to 0xb109600 2024-09-18T01:03:30.128 INFO:teuthology.orchestra.run.smithi164.stdout:2788 punch from 0x1e73600 to 0x1e77c00, (0x4600 bytes) 2024-09-18T01:03:30.132 INFO:teuthology.orchestra.run.smithi164.stdout:2790 trunc from 0xb109600 to 0xcbd8200 2024-09-18T01:03:30.156 INFO:teuthology.orchestra.run.smithi164.stdout:2791 write 0xdeb0600 thru 0xdebf3a8 (0xeda9 bytes) 2024-09-18T01:03:30.156 INFO:teuthology.orchestra.run.smithi164.stdout:2792 punch from 0x98bc00 to 0x990000, (0x4400 bytes) 2024-09-18T01:03:30.158 INFO:teuthology.orchestra.run.smithi164.stdout:2793 read 0xab5c00 thru 0xabd856 (0x7c57 bytes) 2024-09-18T01:03:30.158 INFO:teuthology.orchestra.run.smithi164.stdout:2794 punch from 0xac43e00 to 0xac51600, (0xd800 bytes) 2024-09-18T01:03:30.159 INFO:teuthology.orchestra.run.smithi164.stdout:2795 punch from 0x2c23600 to 0x2c28400, (0x4e00 bytes) 2024-09-18T01:03:30.160 INFO:teuthology.orchestra.run.smithi164.stdout:2796 read 0xa129400 thru 0xa135da2 (0xc9a3 bytes) 2024-09-18T01:03:30.160 INFO:teuthology.orchestra.run.smithi164.stdout:2797 read 0x8565c00 thru 0x8574c8e (0xf08f bytes) 2024-09-18T01:03:30.179 INFO:teuthology.orchestra.run.smithi164.stdout:2802 write 0xdfbc800 thru 0xdfc9758 (0xcf59 bytes) 2024-09-18T01:03:30.180 INFO:teuthology.orchestra.run.smithi164.stdout:2803 write 0xc27c400 thru 0xc283556 (0x7157 bytes) 2024-09-18T01:03:30.180 INFO:teuthology.orchestra.run.smithi164.stdout:2805 read 0x6a16600 thru 0x6a19c24 (0x3625 bytes) 2024-09-18T01:03:30.180 INFO:teuthology.orchestra.run.smithi164.stdout:2806 write 0x8cae00 thru 0x8ce3f5 (0x35f6 bytes) 2024-09-18T01:03:30.181 INFO:teuthology.orchestra.run.smithi164.stdout:2807 write 0x5785a00 thru 0x5786daf (0x13b0 bytes) 2024-09-18T01:03:30.181 INFO:teuthology.orchestra.run.smithi164.stdout:2808 punch from 0xce77200 to 0xce86c00, (0xfa00 bytes) 2024-09-18T01:03:30.182 INFO:teuthology.orchestra.run.smithi164.stdout:2809 write 0xcfce000 thru 0xcfd64d8 (0x84d9 bytes) 2024-09-18T01:03:30.182 INFO:teuthology.orchestra.run.smithi164.stdout:2811 punch from 0x45d1000 to 0x45d9600, (0x8600 bytes) 2024-09-18T01:03:30.182 INFO:teuthology.orchestra.run.smithi164.stdout:2812 write 0x4ab0c00 thru 0x4abec0b (0xe00c bytes) 2024-09-18T01:03:30.186 INFO:teuthology.orchestra.run.smithi164.stdout:2813 trunc from 0xdfc9800 to 0xacdac00 2024-09-18T01:03:30.216 INFO:teuthology.orchestra.run.smithi164.stdout:2814 read 0x1e72000 thru 0x1e80195 (0xe196 bytes) 2024-09-18T01:03:30.218 INFO:teuthology.orchestra.run.smithi164.stdout:2815 read 0x3317800 thru 0x331aa61 (0x3262 bytes) 2024-09-18T01:03:30.218 INFO:teuthology.orchestra.run.smithi164.stdout:2816 punch from 0x3208c00 to 0x3210200, (0x7600 bytes) 2024-09-18T01:03:30.219 INFO:teuthology.orchestra.run.smithi164.stdout:2817 punch from 0x5f71600 to 0x5f7ba00, (0xa400 bytes) 2024-09-18T01:03:30.225 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:30.224+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:30.231 INFO:teuthology.orchestra.run.smithi164.stdout:2819 trunc from 0xacdac00 to 0x11aec00 2024-09-18T01:03:30.259 INFO:teuthology.orchestra.run.smithi164.stdout:2820 read 0x730c00 thru 0x73204d (0x144e bytes) 2024-09-18T01:03:30.272 INFO:teuthology.orchestra.run.smithi164.stdout:2821 write 0x31a3600 thru 0x31acba6 (0x95a7 bytes) 2024-09-18T01:03:30.289 INFO:teuthology.orchestra.run.smithi164.stdout:2823 write 0x61b0c00 thru 0x61b4867 (0x3c68 bytes) 2024-09-18T01:03:30.301 INFO:teuthology.orchestra.run.smithi164.stdout:2824 trunc from 0x61b4a00 to 0xedc0e00 2024-09-18T01:03:30.304 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:30.304 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:30.304 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:30.304 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:30.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:30.315 INFO:teuthology.orchestra.run.smithi164.stdout:2826 read 0xea57400 thru 0xea623f8 (0xaff9 bytes) 2024-09-18T01:03:30.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:30.324 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:30.326 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:30.325+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:30.332 INFO:teuthology.orchestra.run.smithi164.stdout:2827 trunc from 0xedc0e00 to 0xd28000 2024-09-18T01:03:30.347 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:30.359 INFO:teuthology.orchestra.run.smithi164.stdout:2829 trunc from 0xd28000 to 0x4fe8600 2024-09-18T01:03:30.368 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:30.368 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:30.382 INFO:teuthology.orchestra.run.smithi164.stdout:2830 write 0x7b83c00 thru 0x7b89bbd (0x5fbe bytes) 2024-09-18T01:03:30.389 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:30.389 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:30.390 INFO:teuthology.orchestra.run.smithi164.stdout:2831 trunc from 0x7b89c00 to 0x1208e00 2024-09-18T01:03:30.411 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:30.412 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:30.412 INFO:teuthology.orchestra.run.smithi164.stdout:2832 read 0x3a5200 thru 0x3aa096 (0x4e97 bytes) 2024-09-18T01:03:30.426 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:30.425+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:30.431 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:30.431 INFO:teuthology.orchestra.run.smithi164.stdout:2837 write 0xa058a00 thru 0xa05a0e0 (0x16e1 bytes) 2024-09-18T01:03:30.432 INFO:teuthology.orchestra.run.smithi164.stdout:2839 read 0xcfaa00 thru 0xd06e3b (0xc43c bytes) 2024-09-18T01:03:30.433 INFO:teuthology.orchestra.run.smithi164.stdout:2840 write 0x55b5a00 thru 0x55b78e7 (0x1ee8 bytes) 2024-09-18T01:03:30.445 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:30.446 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:30.449 INFO:teuthology.orchestra.run.smithi164.stdout:2842 write 0xa9a0c00 thru 0xa9a8232 (0x7633 bytes) 2024-09-18T01:03:30.453 INFO:teuthology.orchestra.run.smithi164.stdout:2843 read 0x2e29800 thru 0x2e2c20c (0x2a0d bytes) 2024-09-18T01:03:30.453 INFO:teuthology.orchestra.run.smithi164.stdout:2844 punch from 0x421b600 to 0x421d200, (0x1c00 bytes) 2024-09-18T01:03:30.453 INFO:teuthology.orchestra.run.smithi164.stdout:2845 write 0xaeca00 thru 0xafc252 (0xf853 bytes) 2024-09-18T01:03:30.453 INFO:teuthology.orchestra.run.smithi164.stdout:2847 punch from 0x2304c00 to 0x2307a00, (0x2e00 bytes) 2024-09-18T01:03:30.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:30.459 INFO:teuthology.orchestra.run.smithi164.stdout:2848 trunc from 0xa9a8400 to 0x53ae800 2024-09-18T01:03:30.478 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:03:30.491 INFO:teuthology.orchestra.run.smithi164.stdout:2850 write 0x991b600 thru 0x9929138 (0xdb39 bytes) 2024-09-18T01:03:30.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:03:30.498 INFO:teuthology.orchestra.run.smithi164.stdout:2851 trunc from 0x9929200 to 0x67cda00 2024-09-18T01:03:30.535 INFO:teuthology.orchestra.run.smithi164.stdout:2857 write 0xe320400 thru 0xe323894 (0x3495 bytes) 2024-09-18T01:03:30.536 INFO:teuthology.orchestra.run.smithi164.stdout:2858 write 0x25d4000 thru 0x25d9a34 (0x5a35 bytes) 2024-09-18T01:03:30.536 INFO:teuthology.orchestra.run.smithi164.stdout:2859 punch from 0x95fac00 to 0x9601600, (0x6a00 bytes) 2024-09-18T01:03:30.537 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:30.535+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:30.538 INFO:teuthology.orchestra.run.smithi164.stdout:2860 write 0xafb0a00 thru 0xafbbe0a (0xb40b bytes) 2024-09-18T01:03:30.538 INFO:teuthology.orchestra.run.smithi164.stdout:2861 write 0x322d200 thru 0x322fd2e (0x2b2f bytes) 2024-09-18T01:03:30.538 INFO:teuthology.orchestra.run.smithi164.stdout:2862 read 0x34c400 thru 0x350e3a (0x4a3b bytes) 2024-09-18T01:03:30.539 INFO:teuthology.orchestra.run.smithi164.stdout:2867 write 0x6456a00 thru 0x645f33b (0x893c bytes) 2024-09-18T01:03:30.539 INFO:teuthology.orchestra.run.smithi164.stdout:2868 read 0x4027400 thru 0x402861f (0x1220 bytes) 2024-09-18T01:03:30.539 INFO:teuthology.orchestra.run.smithi164.stdout:2870 clone 21 order 25 su 262144 sc 14 2024-09-18T01:03:30.600 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:30.600 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:30.609 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:30.629 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:30.627+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:30.711 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:30.712 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:30.720 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:30.764 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:30.763+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:30.822 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:30.822 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:30.831 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:30.831 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:30.829+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:30.932 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:30.930+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:30.973 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:30.973 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:30.981 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:31.033 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:31.031+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:31.124 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:31.125 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:31.135 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:31.137 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:31.131+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:31.235 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:31.233+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:31.239 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:31.239 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:31.248 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:31.336 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:31.335+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:31.391 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:31.391 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:31.399 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:31.437 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:31.435+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:31.486 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:03:31.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:03:31.538 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:31.537+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:31.545 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:31.545 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:31.554 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:31.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:03:31.599 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone20 from 0xe323a00 (overlap 0xd28000) to 0x150c00 2024-09-18T01:03:31.639 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:31.637+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:31.701 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:31.701 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:31.709 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:31.753 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:31.752+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:31.813 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:31.813 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:31.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:31.863 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:31.862+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:31.930 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:31.930 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:31.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:31.941 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:31.940+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:32.042 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:32.040+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:32.048 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:32.048 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:32.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:32.143 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:32.142+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:32.209 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:32.209 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:32.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:32.244 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:32.243+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:32.325 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:32.325 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:32.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:32.378 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:32.376+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:32.432 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-18T01:03:32.442 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:32.442 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:32.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:32.446 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:32.444+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:32.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:32.487 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:03:32.490 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:03:32.500 INFO:teuthology.orchestra.run.smithi164.stdout:2873 write 0x69d3c00 thru 0x69d5947 (0x1d48 bytes) 2024-09-18T01:03:32.500 INFO:teuthology.orchestra.run.smithi164.stdout:2874 write 0xa072c00 thru 0xa07fb67 (0xcf68 bytes) 2024-09-18T01:03:32.500 INFO:teuthology.orchestra.run.smithi164.stdout:2875 read 0x9ddc200 thru 0x9de7618 (0xb419 bytes) 2024-09-18T01:03:32.505 INFO:teuthology.orchestra.run.smithi164.stdout:2877 write 0xe26d000 thru 0xe270846 (0x3847 bytes) 2024-09-18T01:03:32.505 INFO:teuthology.orchestra.run.smithi164.stdout:2880 punch from 0x5356600 to 0x535b200, (0x4c00 bytes) 2024-09-18T01:03:32.514 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:03:32.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:03:32.526 INFO:teuthology.orchestra.run.smithi164.stdout:2881 write 0x960e000 thru 0x96177e6 (0x97e7 bytes) 2024-09-18T01:03:32.526 INFO:teuthology.orchestra.run.smithi164.stdout:2882 read 0x29eb800 thru 0x29f798b (0xc18c bytes) 2024-09-18T01:03:32.536 INFO:teuthology.orchestra.run.smithi164.stdout:2883 trunc from 0xe323a00 to 0x604e000 2024-09-18T01:03:32.547 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:32.546+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:32.600 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:32.601 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:32.624 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:32.624 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:32.626 INFO:teuthology.orchestra.run.smithi164.stdout:2886 punch from 0x15abc00 to 0x15b8000, (0xc400 bytes) 2024-09-18T01:03:32.628 INFO:teuthology.orchestra.run.smithi164.stdout:2887 punch from 0x47a9c00 to 0x47ab400, (0x1800 bytes) 2024-09-18T01:03:32.636 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 44.357493728999998, 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 44.352867826000001, 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 44.352789776999998, 2024-09-18T01:03:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1292 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1774592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.296018052000001, 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050913359999999998, 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1292 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338601+0000", 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.024e-06 2024-09-18T01:03:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338610+0000", 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5469999999999997e-06 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338616+0000", 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389393+0000", 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05077661 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389472+0000", 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9776000000000003e-05 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389512+0000", 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0000000000000003e-05 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.295993734, 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.295967869999998, 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.29594539, 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.295924354, 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.29589696, 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.295872654999997, 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292473172000001, 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292448612000001, 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292407095000002, 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292386593000003, 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292366127000001, 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292324786999998, 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292304749000003, 2024-09-18T01:03:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292262418, 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292241425999997, 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.292190456, 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:32.742 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:32.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:32.744 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:32.745 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:32.745 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:32.746 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:32.747 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:32.747 INFO:teuthology.orchestra.run.smithi164.stdout:2888 write 0xc17a600 thru 0xc18441e (0x9e1f bytes) 2024-09-18T01:03:32.747 INFO:teuthology.orchestra.run.smithi164.stdout:2890 read 0x5340c00 thru 0x53490c3 (0x84c4 bytes) 2024-09-18T01:03:32.747 INFO:teuthology.orchestra.run.smithi164.stdout:2891 write 0x3716800 thru 0x3722b2c (0xc32d bytes) 2024-09-18T01:03:32.747 INFO:teuthology.orchestra.run.smithi164.stdout:2893 read 0x2826200 thru 0x2833888 (0xd689 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2894 read 0xafe8e00 thru 0xafec509 (0x370a bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2896 read 0x7132400 thru 0x7137417 (0x5018 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2897 punch from 0x3b92200 to 0x3b94400, (0x2200 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2898 read 0x721400 thru 0x7235f6 (0x21f7 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2899 read 0x11ed800 thru 0x11ef07f (0x1880 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2900 read 0x5e9fa00 thru 0x5ea8e65 (0x9466 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2901 write 0xdd55c00 thru 0xdd60652 (0xaa53 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2902 read 0xb3e5000 thru 0xb3ee2ac (0x92ad bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2903 read 0xc764800 thru 0xc77476c (0xff6d bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2905 write 0x796800 thru 0x7a01bf (0x99c0 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2906 write 0xa01600 thru 0xa09c44 (0x8645 bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2909 read 0xd90ce00 thru 0xd90fe4a (0x304b bytes) 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2910 trunc from 0xdd60800 to 0xe3f0200 2024-09-18T01:03:32.748 INFO:teuthology.orchestra.run.smithi164.stdout:2912 read 0xe0da000 thru 0xe0de02d (0x402e bytes) 2024-09-18T01:03:32.749 INFO:teuthology.orchestra.run.smithi164.stdout:2914 read 0xdba1800 thru 0xdbacfd9 (0xb7da bytes) 2024-09-18T01:03:32.749 INFO:teuthology.orchestra.run.smithi164.stdout:2915 read 0x9062200 thru 0x906519b (0x2f9c bytes) 2024-09-18T01:03:32.749 INFO:teuthology.orchestra.run.smithi164.stdout:2916 trunc from 0xe3f0200 to 0x962ba00 2024-09-18T01:03:32.749 INFO:teuthology.orchestra.run.smithi164.stdout:2918 read 0x2bf7800 thru 0x2c03a15 (0xc216 bytes) 2024-09-18T01:03:32.749 INFO:teuthology.orchestra.run.smithi164.stdout:2919 write 0xdccb600 thru 0xdccb7ad (0x1ae bytes) 2024-09-18T01:03:32.749 INFO:teuthology.orchestra.run.smithi164.stdout:2920 read 0x67b3400 thru 0x67b3a7b (0x67c bytes) 2024-09-18T01:03:32.749 INFO:teuthology.orchestra.run.smithi164.stdout:2923 clone 22 order 18 su 65536 sc 11 2024-09-18T01:03:32.749 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:32.739+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:32.842 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:32.840+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:32.885 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:03:32.891 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:32.891 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:32.896 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-18T01:03:32.487252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":735574725,"num_objects":869,"num_object_clones":170,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":323,"num_read":157355,"num_read_kb":724033,"num_write":12092,"num_write_kb":183982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3303085,"num_keys_recovered":14,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10193,"ondisk_log_size":10193,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":244340,"kb_used_data":31492,"kb_used_omap":7,"kb_used_meta":212728,"kb_avail":471614860,"statfs":{"total":483183820800,"available":482933616640,"internally_reserved":0,"allocated":32247808,"data_stored":241059600,"data_compressed":5620447,"data_compressed_allocated":19152896,"data_compressed_original":232193024,"omap_allocated":7973,"internal_metadata":217833691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78099346,"num_objects":118,"num_object_clones":21,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":25,"num_read":14503,"num_read_kb":151200,"num_write":1084,"num_write_kb":30386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024065"},"pg_stats":[{"pgid":"3.1f","version":"59'161","reported_seq":3067,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.899025+0000","last_change":"2024-09-18T01:02:43.469611+0000","last_active":"2024-09-18T01:03:28.899025+0000","last_peered":"2024-09-18T01:03:28.899025+0000","last_clean":"2024-09-18T01:03:28.899025+0000","last_became_active":"2024-09-18T01:02:41.451184+0000","last_became_peered":"2024-09-18T01:02:41.451184+0000","last_unstale":"2024-09-18T01:03:28.899025+0000","last_undegraded":"2024-09-18T01:03:28.899025+0000","last_fullsized":"2024-09-18T01:03:28.899025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:50.009888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005167,"stat_sum":{"num_bytes":40245776,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":2854,"num_read_kb":33535,"num_write":207,"num_write_kb":6984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"59'244","reported_seq":1008,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.757415+0000","last_change":"2024-09-18T01:02:43.469963+0000","last_active":"2024-09-18T01:03:28.757415+0000","last_peered":"2024-09-18T01:03:28.757415+0000","last_clean":"2024-09-18T01:03:28.757415+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:03:28.757415+0000","last_undegraded":"2024-09-18T01:03:28.757415+0000","last_fullsized":"2024-09-18T01:03:28.757415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:03.955406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013026730000000001,"stat_sum":{"num_bytes":5356729,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":953,"num_read_kb":1584,"num_write":277,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"59'219","reported_seq":5411,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.632259+0000","last_change":"2024-09-18T01:02:49.322327+0000","last_active":"2024-09-18T01:03:28.632259+0000","last_peered":"2024-09-18T01:03:28.632259+0000","last_clean":"2024-09-18T01:03:28.632259+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:03:28.632259+0000","last_undegraded":"2024-09-18T01:03:28.632259+0000","last_fullsized":"2024-09-18T01:03:28.632259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:00.299907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918460000000001,"stat_sum":{"num_bytes":13355232,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":5948,"num_read_kb":17787,"num_write":214,"num_write_kb":6280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"59'356","reported_seq":5638,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.913840+0000","last_change":"2024-09-18T01:02:49.301865+0000","last_active":"2024-09-18T01:03:28.913840+0000","last_peered":"2024-09-18T01:03:28.913840+0000","last_clean":"2024-09-18T01:03:28.913840+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:03:28.913840+0000","last_undegraded":"2024-09-18T01:03:28.913840+0000","last_fullsized":"2024-09-18T01:03:28.913840+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:48:41.706555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242879999999999,"stat_sum":{"num_bytes":33742440,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":6686,"num_read_kb":27766,"num_write":366,"num_write_kb":7700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"59'323","reported_seq":2271,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.892464+0000","last_change":"2024-09-18T01:02:43.469715+0000","last_active":"2024-09-18T01:03:28.892464+0000","last_peered":"2024-09-18T01:03:28.892464+0000","last_clean":"2024-09-18T01:03:28.892464+0000","last_became_active":"2024-09-18T01:02:41.454655+0000","last_became_peered":"2024-09-18T01:02:41.454655+0000","last_unstale":"2024-09-18T01:03:28.892464+0000","last_undegraded":"2024-09-18T01:03:28.892464+0000","last_fullsized":"2024-09-18T01:03:28.892464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:58.818196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087094899999999999,"stat_sum":{"num_bytes":13982305,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":2542,"num_read_kb":8903,"num_write":450,"num_write_kb":3726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"59'152","reported_seq":2812,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.885763+0000","last_change":"2024-09-18T01:02:43.474216+0000","last_active":"2024-09-18T01:03:28.885763+0000","last_peered":"2024-09-18T01:03:28.885763+0000","last_clean":"2024-09-18T01:03:28.885763+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:03:28.885763+0000","last_undegraded":"2024-09-18T01:03:28.885763+0000","last_fullsized":"2024-09-18T01:03:28.885763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:59.750926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074574000000000005,"stat_sum":{"num_bytes":5772016,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":2604,"num_read_kb":7829,"num_write":152,"num_write_kb":6184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"59'477","reported_seq":6674,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906463+0000","last_change":"2024-09-18T01:02:43.475997+0000","last_active":"2024-09-18T01:03:28.906463+0000","last_peered":"2024-09-18T01:03:28.906463+0000","last_clean":"2024-09-18T01:03:28.906463+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:03:28.906463+0000","last_undegraded":"2024-09-18T01:03:28.906463+0000","last_fullsized":"2024-09-18T01:03:28.906463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:52:14.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038270300000000003,"stat_sum":{"num_bytes":28103753,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":8753,"num_read_kb":34604,"num_write":465,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"59'287","reported_seq":3871,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.564756+0000","last_change":"2024-09-18T01:02:43.468721+0000","last_active":"2024-09-18T01:03:28.564756+0000","last_peered":"2024-09-18T01:03:28.564756+0000","last_clean":"2024-09-18T01:03:28.564756+0000","last_became_active":"2024-09-18T01:02:41.459321+0000","last_became_peered":"2024-09-18T01:02:41.459321+0000","last_unstale":"2024-09-18T01:03:28.564756+0000","last_undegraded":"2024-09-18T01:03:28.564756+0000","last_fullsized":"2024-09-18T01:03:28.564756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:13.145663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061623500000000002,"stat_sum":{"num_bytes":47937061,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":3888,"num_read_kb":59080,"num_write":331,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"59'511","reported_seq":4215,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.761661+0000","last_change":"2024-09-18T01:02:49.315232+0000","last_active":"2024-09-18T01:03:28.761661+0000","last_peered":"2024-09-18T01:03:28.761661+0000","last_clean":"2024-09-18T01:03:28.761661+0000","last_became_active":"2024-09-18T01:02:49.293095+0000","last_became_peered":"2024-09-18T01:02:49.293095+0000","last_unstale":"2024-09-18T01:03:28.761661+0000","last_undegraded":"2024-09-18T01:03:28.761661+0000","last_fullsized":"2024-09-18T01:03:28.761661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:17.702565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097728599999999995,"stat_sum":{"num_bytes":27739725,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5287,"num_read_kb":17711,"num_write":761,"num_write_kb":2931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"59'219","reported_seq":2142,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.775439+0000","last_change":"2024-09-18T01:03:21.299624+0000","last_active":"2024-09-18T01:03:28.775439+0000","last_peered":"2024-09-18T01:03:28.775439+0000","last_clean":"2024-09-18T01:03:28.775439+0000","last_became_active":"2024-09-18T01:03:21.298726+0000","last_became_peered":"2024-09-18T01:03:21.298726+0000","last_unstale":"2024-09-18T01:03:28.775439+0000","last_undegraded":"2024-09-18T01:03:28.775439+0000","last_fullsized":"2024-09-18T01:03:28.775439+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:26.762981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099035500000000005,"stat_sum":{"num_bytes":14968494,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":2070,"num_read_kb":19451,"num_write":267,"num_write_kb":5646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":3286625,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"59'176","reported_seq":4649,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.913940+0000","last_change":"2024-09-18T01:02:43.464371+0000","last_active":"2024-09-18T01:03:28.913940+0000","last_peered":"2024-09-18T01:03:28.913940+0000","last_clean":"2024-09-18T01:03:28.913940+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:03:28.913940+0000","last_undegraded":"2024-09-18T01:03:28.913940+0000","last_fullsized":"2024-09-18T01:03:28.913940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:20:34.561443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077757900000000003,"stat_sum":{"num_bytes":19914373,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":4443,"num_read_kb":22697,"num_write":160,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"59'376","reported_seq":3560,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.892471+0000","last_change":"2024-09-18T01:02:43.464338+0000","last_active":"2024-09-18T01:03:28.892471+0000","last_peered":"2024-09-18T01:03:28.892471+0000","last_clean":"2024-09-18T01:03:28.892471+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:03:28.892471+0000","last_undegraded":"2024-09-18T01:03:28.892471+0000","last_fullsized":"2024-09-18T01:03:28.892471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:24.155448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080305299999999997,"stat_sum":{"num_bytes":11473500,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":4720,"num_read_kb":5951,"num_write":435,"num_write_kb":1294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479546+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:03:28.479546+0000","last_peered":"2024-09-18T01:03:28.479546+0000","last_clean":"2024-09-18T01:03:28.479546+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:03:28.479546+0000","last_undegraded":"2024-09-18T01:03:28.479546+0000","last_fullsized":"2024-09-18T01:03:28.479546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"59'287","reported_seq":5186,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906478+0000","last_change":"2024-09-18T01:02:43.469252+0000","last_active":"2024-09-18T01:03:28.906478+0000","last_peered":"2024-09-18T01:03:28.906478+0000","last_clean":"2024-09-18T01:03:28.906478+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:03:28.906478+0000","last_undegraded":"2024-09-18T01:03:28.906478+0000","last_fullsized":"2024-09-18T01:03:28.906478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:52.498307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066086499999999996,"stat_sum":{"num_bytes":79278629,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":5452,"num_read_kb":64068,"num_write":374,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.467157+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:03:28.467157+0000","last_peered":"2024-09-18T01:03:28.467157+0000","last_clean":"2024-09-18T01:03:28.467157+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:03:28.467157+0000","last_undegraded":"2024-09-18T01:03:28.467157+0000","last_fullsized":"2024-09-18T01:03:28.467157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"58'1578","reported_seq":7474,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.774811+0000","last_change":"2024-09-18T01:02:43.469995+0000","last_active":"2024-09-18T01:03:28.774811+0000","last_peered":"2024-09-18T01:03:28.774811+0000","last_clean":"2024-09-18T01:03:28.774811+0000","last_became_active":"2024-09-18T01:02:41.456801+0000","last_became_peered":"2024-09-18T01:02:41.456801+0000","last_unstale":"2024-09-18T01:03:28.774811+0000","last_undegraded":"2024-09-18T01:03:28.774811+0000","last_fullsized":"2024-09-18T01:03:28.774811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1578,"log_dups_size":0,"ondisk_log_size":1578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:53:50.463232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159869,"stat_sum":{"num_bytes":25973986,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":14566,"num_read_kb":30805,"num_write":2683,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479533+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:03:28.479533+0000","last_peered":"2024-09-18T01:03:28.479533+0000","last_clean":"2024-09-18T01:03:28.479533+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:03:28.479533+0000","last_undegraded":"2024-09-18T01:03:28.479533+0000","last_fullsized":"2024-09-18T01:03:28.479533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"59'179","reported_seq":3583,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.898386+0000","last_change":"2024-09-18T01:02:43.476334+0000","last_active":"2024-09-18T01:03:28.898386+0000","last_peered":"2024-09-18T01:03:28.898386+0000","last_clean":"2024-09-18T01:03:28.898386+0000","last_became_active":"2024-09-18T01:02:41.457022+0000","last_became_peered":"2024-09-18T01:02:41.457022+0000","last_unstale":"2024-09-18T01:03:28.898386+0000","last_undegraded":"2024-09-18T01:03:28.898386+0000","last_fullsized":"2024-09-18T01:03:28.898386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:35.329583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050705100000000001,"stat_sum":{"num_bytes":10953812,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":3710,"num_read_kb":6800,"num_write":166,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.467075+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:03:28.467075+0000","last_peered":"2024-09-18T01:03:28.467075+0000","last_clean":"2024-09-18T01:03:28.467075+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:03:28.467075+0000","last_undegraded":"2024-09-18T01:03:28.467075+0000","last_fullsized":"2024-09-18T01:03:28.467075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"59'181","reported_seq":6078,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.899074+0000","last_change":"2024-09-18T01:02:49.294263+0000","last_active":"2024-09-18T01:03:28.899074+0000","last_peered":"2024-09-18T01:03:28.899074+0000","last_clean":"2024-09-18T01:03:28.899074+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:03:28.899074+0000","last_undegraded":"2024-09-18T01:03:28.899074+0000","last_fullsized":"2024-09-18T01:03:28.899074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:55.474675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22912544,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":5881,"num_read_kb":32468,"num_write":204,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'404","reported_seq":5357,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.900504+0000","last_change":"2024-09-18T01:02:43.473795+0000","last_active":"2024-09-18T01:03:28.900504+0000","last_peered":"2024-09-18T01:03:28.900504+0000","last_clean":"2024-09-18T01:03:28.900504+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:03:28.900504+0000","last_undegraded":"2024-09-18T01:03:28.900504+0000","last_fullsized":"2024-09-18T01:03:28.900504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:19.793410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031347599999999998,"stat_sum":{"num_bytes":42097305,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":6750,"num_read_kb":44974,"num_write":449,"num_write_kb":5570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.467245+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:03:28.467245+0000","last_peered":"2024-09-18T01:03:28.467245+0000","last_clean":"2024-09-18T01:03:28.467245+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:03:28.467245+0000","last_undegraded":"2024-09-18T01:03:28.467245+0000","last_fullsized":"2024-09-18T01:03:28.467245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'159","reported_seq":4247,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.773315+0000","last_change":"2024-09-18T01:02:43.476102+0000","last_active":"2024-09-18T01:03:28.773315+0000","last_peered":"2024-09-18T01:03:28.773315+0000","last_clean":"2024-09-18T01:03:28.773315+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:03:28.773315+0000","last_undegraded":"2024-09-18T01:03:28.773315+0000","last_fullsized":"2024-09-18T01:03:28.773315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:01.384338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043811300000000001,"stat_sum":{"num_bytes":35942146,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":4069,"num_read_kb":24616,"num_write":167,"num_write_kb":5011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.481572+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:03:28.481572+0000","last_peered":"2024-09-18T01:03:28.481572+0000","last_clean":"2024-09-18T01:03:28.481572+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:03:28.481572+0000","last_undegraded":"2024-09-18T01:03:28.481572+0000","last_fullsized":"2024-09-18T01:03:28.481572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'242","reported_seq":3457,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.779540+0000","last_change":"2024-09-18T01:02:43.468762+0000","last_active":"2024-09-18T01:03:28.779540+0000","last_peered":"2024-09-18T01:03:28.779540+0000","last_clean":"2024-09-18T01:03:28.779540+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:03:28.779540+0000","last_undegraded":"2024-09-18T01:03:28.779540+0000","last_fullsized":"2024-09-18T01:03:28.779540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:47:06.064710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296199999999997,"stat_sum":{"num_bytes":22983680,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":4050,"num_read_kb":24657,"num_write":213,"num_write_kb":7184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479228+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:03:28.479228+0000","last_peered":"2024-09-18T01:03:28.479228+0000","last_clean":"2024-09-18T01:03:28.479228+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:03:28.479228+0000","last_undegraded":"2024-09-18T01:03:28.479228+0000","last_fullsized":"2024-09-18T01:03:28.479228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"59'284","reported_seq":3694,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.772934+0000","last_change":"2024-09-18T01:02:49.337150+0000","last_active":"2024-09-18T01:03:28.772934+0000","last_peered":"2024-09-18T01:03:28.772934+0000","last_clean":"2024-09-18T01:03:28.772934+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:03:28.772934+0000","last_undegraded":"2024-09-18T01:03:28.772934+0000","last_fullsized":"2024-09-18T01:03:28.772934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:12.155435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049608300000000003,"stat_sum":{"num_bytes":8573077,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":4466,"num_read_kb":22223,"num_write":253,"num_write_kb":4370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":79,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.481696+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:03:28.481696+0000","last_peered":"2024-09-18T01:03:28.481696+0000","last_clean":"2024-09-18T01:03:28.481696+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:03:28.481696+0000","last_undegraded":"2024-09-18T01:03:28.481696+0000","last_fullsized":"2024-09-18T01:03:28.481696+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":253,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479235+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:03:28.479235+0000","last_peered":"2024-09-18T01:03:28.479235+0000","last_clean":"2024-09-18T01:03:28.479235+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:03:28.479235+0000","last_undegraded":"2024-09-18T01:03:28.479235+0000","last_fullsized":"2024-09-18T01:03:28.479235+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"59'204","reported_seq":4502,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.898145+0000","last_change":"2024-09-18T01:02:43.468997+0000","last_active":"2024-09-18T01:03:28.898145+0000","last_peered":"2024-09-18T01:03:28.898145+0000","last_clean":"2024-09-18T01:03:28.898145+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:03:28.898145+0000","last_undegraded":"2024-09-18T01:03:28.898145+0000","last_fullsized":"2024-09-18T01:03:28.898145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:44:04.566739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063056400000000002,"stat_sum":{"num_bytes":23992937,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":4519,"num_read_kb":16351,"num_write":234,"num_write_kb":8930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"59'247","reported_seq":3659,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.561513+0000","last_change":"2024-09-18T01:02:43.474261+0000","last_active":"2024-09-18T01:03:28.561513+0000","last_peered":"2024-09-18T01:03:28.561513+0000","last_clean":"2024-09-18T01:03:28.561513+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:03:28.561513+0000","last_undegraded":"2024-09-18T01:03:28.561513+0000","last_fullsized":"2024-09-18T01:03:28.561513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:50.058657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065863900000000003,"stat_sum":{"num_bytes":15702558,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":3727,"num_read_kb":20808,"num_write":302,"num_write_kb":9105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"59'531","reported_seq":3628,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.557837+0000","last_change":"2024-09-18T01:02:43.464113+0000","last_active":"2024-09-18T01:03:28.557837+0000","last_peered":"2024-09-18T01:03:28.557837+0000","last_clean":"2024-09-18T01:03:28.557837+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:03:28.557837+0000","last_undegraded":"2024-09-18T01:03:28.557837+0000","last_fullsized":"2024-09-18T01:03:28.557837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:56.079458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066269099999999997,"stat_sum":{"num_bytes":13186186,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":5140,"num_read_kb":12829,"num_write":711,"num_write_kb":2229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"59'319","reported_seq":4177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906375+0000","last_change":"2024-09-18T01:02:43.476485+0000","last_active":"2024-09-18T01:03:28.906375+0000","last_peered":"2024-09-18T01:03:28.906375+0000","last_clean":"2024-09-18T01:03:28.906375+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:03:28.906375+0000","last_undegraded":"2024-09-18T01:03:28.906375+0000","last_fullsized":"2024-09-18T01:03:28.906375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:26.218489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054801899999999996,"stat_sum":{"num_bytes":22089760,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":5374,"num_read_kb":28757,"num_write":268,"num_write_kb":5624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"59'161","reported_seq":3184,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.793063+0000","last_change":"2024-09-18T01:02:49.302821+0000","last_active":"2024-09-18T01:03:28.793063+0000","last_peered":"2024-09-18T01:03:28.793063+0000","last_clean":"2024-09-18T01:03:28.793063+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:03:28.793063+0000","last_undegraded":"2024-09-18T01:03:28.793063+0000","last_fullsized":"2024-09-18T01:03:28.793063+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:45.502789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043103,"stat_sum":{"num_bytes":14301184,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":2929,"num_read_kb":17165,"num_write":158,"num_write_kb":3822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"59'241","reported_seq":3891,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.776643+0000","last_change":"2024-09-18T01:02:43.464136+0000","last_active":"2024-09-18T01:03:28.776643+0000","last_peered":"2024-09-18T01:03:28.776643+0000","last_clean":"2024-09-18T01:03:28.776643+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:03:28.776643+0000","last_undegraded":"2024-09-18T01:03:28.776643+0000","last_fullsized":"2024-09-18T01:03:28.776643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:55.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061418400000000004,"stat_sum":{"num_bytes":13560832,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":4352,"num_read_kb":13338,"num_write":218,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"59'189","reported_seq":4731,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.911247+0000","last_change":"2024-09-18T01:02:49.312596+0000","last_active":"2024-09-18T01:03:28.911247+0000","last_peered":"2024-09-18T01:03:28.911247+0000","last_clean":"2024-09-18T01:03:28.911247+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:03:28.911247+0000","last_undegraded":"2024-09-18T01:03:28.911247+0000","last_fullsized":"2024-09-18T01:03:28.911247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:02:40.082959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013236470000000001,"stat_sum":{"num_bytes":30078513,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":4593,"num_read_kb":23323,"num_write":168,"num_write_kb":6694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"59'382","reported_seq":11200,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.785863+0000","last_change":"2024-09-18T01:02:49.321182+0000","last_active":"2024-09-18T01:03:28.785863+0000","last_peered":"2024-09-18T01:03:28.785863+0000","last_clean":"2024-09-18T01:03:28.785863+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:03:28.785863+0000","last_undegraded":"2024-09-18T01:03:28.785863+0000","last_fullsized":"2024-09-18T01:03:28.785863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:56.098763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011817170000000001,"stat_sum":{"num_bytes":8214543,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":9347,"num_read_kb":15854,"num_write":240,"num_write_kb":3235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"59'187","reported_seq":2194,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.899472+0000","last_change":"2024-09-18T01:02:49.345428+0000","last_active":"2024-09-18T01:03:28.899472+0000","last_peered":"2024-09-18T01:03:28.899472+0000","last_clean":"2024-09-18T01:03:28.899472+0000","last_became_active":"2024-09-18T01:02:49.296123+0000","last_became_peered":"2024-09-18T01:02:49.296123+0000","last_unstale":"2024-09-18T01:03:28.899472+0000","last_undegraded":"2024-09-18T01:03:28.899472+0000","last_fullsized":"2024-09-18T01:03:28.899472+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:33.113422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053670300000000003,"stat_sum":{"num_bytes":17298696,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":2447,"num_read_kb":14495,"num_write":167,"num_write_kb":2630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"59'177","reported_seq":2704,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.897886+0000","last_change":"2024-09-18T01:02:43.463798+0000","last_active":"2024-09-18T01:03:28.897886+0000","last_peered":"2024-09-18T01:03:28.897886+0000","last_clean":"2024-09-18T01:03:28.897886+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:03:28.897886+0000","last_undegraded":"2024-09-18T01:03:28.897886+0000","last_fullsized":"2024-09-18T01:03:28.897886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:49.447708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042487699999999999,"stat_sum":{"num_bytes":7055872,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":2884,"num_read_kb":10827,"num_write":161,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"59'455","reported_seq":7706,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.679660+0000","last_change":"2024-09-18T01:02:43.468742+0000","last_active":"2024-09-18T01:03:28.679660+0000","last_peered":"2024-09-18T01:03:28.679660+0000","last_clean":"2024-09-18T01:03:28.679660+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:03:28.679660+0000","last_undegraded":"2024-09-18T01:03:28.679660+0000","last_fullsized":"2024-09-18T01:03:28.679660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:25:48.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043475499999999999,"stat_sum":{"num_bytes":38362802,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":9296,"num_read_kb":31449,"num_write":545,"num_write_kb":12457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'165","reported_seq":3165,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906375+0000","last_change":"2024-09-18T01:02:49.356374+0000","last_active":"2024-09-18T01:03:28.906375+0000","last_peered":"2024-09-18T01:03:28.906375+0000","last_clean":"2024-09-18T01:03:28.906375+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:03:28.906375+0000","last_undegraded":"2024-09-18T01:03:28.906375+0000","last_fullsized":"2024-09-18T01:03:28.906375+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:22.333692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311600000000002,"stat_sum":{"num_bytes":19833872,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":2938,"num_read_kb":11227,"num_write":165,"num_write_kb":5592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":734984338,"num_objects":865,"num_object_clones":170,"num_object_copies":1730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":865,"num_whiteouts":323,"num_read":157238,"num_read_kb":723932,"num_write":11931,"num_write_kb":181564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3303066,"num_keys_recovered":14,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28958720,"data_stored":237762852,"data_compressed":5582956,"data_compressed_allocated":19038208,"data_compressed_original":230079488,"omap_allocated":0,"internal_metadata":0},"log_size":10073,"ondisk_log_size":10073,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640271,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640273,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49392,"kb_used_data":5728,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94322448,"statfs":{"total":96636764160,"available":96586186752,"internally_reserved":0,"allocated":5865472,"data_stored":42484418,"data_compressed":1007932,"data_compressed_allocated":3436544,"data_compressed_original":40852992,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45288,"kb_used_data":5784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326552,"statfs":{"total":96636764160,"available":96590389248,"internally_reserved":0,"allocated":5922816,"data_stored":41846576,"data_compressed":1042185,"data_compressed_allocated":3325952,"data_compressed_original":40109056,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640271,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62860,"kb_used_data":5304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308980,"statfs":{"total":96636764160,"available":96572395520,"internally_reserved":0,"allocated":5431296,"data_stored":42227394,"data_compressed":751685,"data_compressed_allocated":3158016,"data_compressed_original":40742912,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640273,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51996,"kb_used_data":6896,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319844,"statfs":{"total":96636764160,"available":96583520256,"internally_reserved":0,"allocated":7061504,"data_stored":54209882,"data_compressed":1312067,"data_compressed_allocated":4354048,"data_compressed_original":52379136,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34804,"kb_used_data":7780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337036,"statfs":{"total":96636764160,"available":96601124864,"internally_reserved":0,"allocated":7966720,"data_stored":60291330,"data_compressed":1506578,"data_compressed_allocated":4878336,"data_compressed_original":58108928,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7229440,"data_stored":59433661,"data_compressed":1498028,"data_compressed_allocated":4841472,"data_compressed_original":57535488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6434816,"data_stored":53007669,"data_compressed":1291140,"data_compressed_allocated":4280320,"data_compressed_original":51346944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4829184,"data_stored":42025498,"data_compressed":751819,"data_compressed_allocated":3166208,"data_compressed_original":40759296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5275648,"data_stored":41078888,"data_compressed":1034037,"data_compressed_allocated":3313664,"data_compressed_original":39584768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":42217136,"data_compressed":1007932,"data_compressed_allocated":3436544,"data_compressed_original":40852992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:03:32.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:03:32.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:32.943 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:32.942+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:33.045 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:33.044+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:33.051 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:33.052 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:33.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:33.146 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:33.145+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.436198234999999, 2024-09-18T01:03:33.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:33.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:33.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:33.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.436178277, 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:33.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:33.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.436159335999999, 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:33.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:33.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.436137827, 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.436112098999999, 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:33.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427380240000002, 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:33.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427357526000002, 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:33.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427338396, 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:33.216 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:33.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.42731955, 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427300672000001, 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:33.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427278807, 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:33.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:33.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427255171999999, 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:33.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:33.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427221522, 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:33.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:33.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:33.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:33.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427193667000001, 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:33.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:33.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427173843999999, 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:33.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:33.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427153409999999, 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:33.266 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:33.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:33.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:33.272 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427131330000002, 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:33.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427109007999999, 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:33.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:33.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:33.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427086612, 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:33.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:33.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:33.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.427066193000002, 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:33.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:33.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:33.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:33.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.297 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:33.299 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:33.300 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:33.301 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:03:33.302 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-18T01:03:32.487252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":735574725,"num_objects":869,"num_object_clones":170,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":323,"num_read":157355,"num_read_kb":724033,"num_write":12092,"num_write_kb":183982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3303085,"num_keys_recovered":14,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10193,"ondisk_log_size":10193,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":244340,"kb_used_data":31492,"kb_used_omap":7,"kb_used_meta":212728,"kb_avail":471614860,"statfs":{"total":483183820800,"available":482933616640,"internally_reserved":0,"allocated":32247808,"data_stored":241059600,"data_compressed":5620447,"data_compressed_allocated":19152896,"data_compressed_original":232193024,"omap_allocated":7973,"internal_metadata":217833691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78099346,"num_objects":118,"num_object_clones":21,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":25,"num_read":14503,"num_read_kb":151200,"num_write":1084,"num_write_kb":30386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024065"},"pg_stats":[{"pgid":"3.1f","version":"59'161","reported_seq":3067,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.899025+0000","last_change":"2024-09-18T01:02:43.469611+0000","last_active":"2024-09-18T01:03:28.899025+0000","last_peered":"2024-09-18T01:03:28.899025+0000","last_clean":"2024-09-18T01:03:28.899025+0000","last_became_active":"2024-09-18T01:02:41.451184+0000","last_became_peered":"2024-09-18T01:02:41.451184+0000","last_unstale":"2024-09-18T01:03:28.899025+0000","last_undegraded":"2024-09-18T01:03:28.899025+0000","last_fullsized":"2024-09-18T01:03:28.899025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:17:50.009888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005167,"stat_sum":{"num_bytes":40245776,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":2854,"num_read_kb":33535,"num_write":207,"num_write_kb":6984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"59'244","reported_seq":1008,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.757415+0000","last_change":"2024-09-18T01:02:43.469963+0000","last_active":"2024-09-18T01:03:28.757415+0000","last_peered":"2024-09-18T01:03:28.757415+0000","last_clean":"2024-09-18T01:03:28.757415+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:03:28.757415+0000","last_undegraded":"2024-09-18T01:03:28.757415+0000","last_fullsized":"2024-09-18T01:03:28.757415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:03.955406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013026730000000001,"stat_sum":{"num_bytes":5356729,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":953,"num_read_kb":1584,"num_write":277,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"59'219","reported_seq":5411,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.632259+0000","last_change":"2024-09-18T01:02:49.322327+0000","last_active":"2024-09-18T01:03:28.632259+0000","last_peered":"2024-09-18T01:03:28.632259+0000","last_clean":"2024-09-18T01:03:28.632259+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:03:28.632259+0000","last_undegraded":"2024-09-18T01:03:28.632259+0000","last_fullsized":"2024-09-18T01:03:28.632259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:00.299907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918460000000001,"stat_sum":{"num_bytes":13355232,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":5948,"num_read_kb":17787,"num_write":214,"num_write_kb":6280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"59'356","reported_seq":5638,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.913840+0000","last_change":"2024-09-18T01:02:49.301865+0000","last_active":"2024-09-18T01:03:28.913840+0000","last_peered":"2024-09-18T01:03:28.913840+0000","last_clean":"2024-09-18T01:03:28.913840+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:03:28.913840+0000","last_undegraded":"2024-09-18T01:03:28.913840+0000","last_fullsized":"2024-09-18T01:03:28.913840+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:48:41.706555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010242879999999999,"stat_sum":{"num_bytes":33742440,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":6686,"num_read_kb":27766,"num_write":366,"num_write_kb":7700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"59'323","reported_seq":2271,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.892464+0000","last_change":"2024-09-18T01:02:43.469715+0000","last_active":"2024-09-18T01:03:28.892464+0000","last_peered":"2024-09-18T01:03:28.892464+0000","last_clean":"2024-09-18T01:03:28.892464+0000","last_became_active":"2024-09-18T01:02:41.454655+0000","last_became_peered":"2024-09-18T01:02:41.454655+0000","last_unstale":"2024-09-18T01:03:28.892464+0000","last_undegraded":"2024-09-18T01:03:28.892464+0000","last_fullsized":"2024-09-18T01:03:28.892464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:58.818196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087094899999999999,"stat_sum":{"num_bytes":13982305,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":2542,"num_read_kb":8903,"num_write":450,"num_write_kb":3726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"59'152","reported_seq":2812,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.885763+0000","last_change":"2024-09-18T01:02:43.474216+0000","last_active":"2024-09-18T01:03:28.885763+0000","last_peered":"2024-09-18T01:03:28.885763+0000","last_clean":"2024-09-18T01:03:28.885763+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:03:28.885763+0000","last_undegraded":"2024-09-18T01:03:28.885763+0000","last_fullsized":"2024-09-18T01:03:28.885763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:59.750926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074574000000000005,"stat_sum":{"num_bytes":5772016,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":2604,"num_read_kb":7829,"num_write":152,"num_write_kb":6184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"59'477","reported_seq":6674,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906463+0000","last_change":"2024-09-18T01:02:43.475997+0000","last_active":"2024-09-18T01:03:28.906463+0000","last_peered":"2024-09-18T01:03:28.906463+0000","last_clean":"2024-09-18T01:03:28.906463+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:03:28.906463+0000","last_undegraded":"2024-09-18T01:03:28.906463+0000","last_fullsized":"2024-09-18T01:03:28.906463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:52:14.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038270300000000003,"stat_sum":{"num_bytes":28103753,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":8753,"num_read_kb":34604,"num_write":465,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"59'287","reported_seq":3871,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.564756+0000","last_change":"2024-09-18T01:02:43.468721+0000","last_active":"2024-09-18T01:03:28.564756+0000","last_peered":"2024-09-18T01:03:28.564756+0000","last_clean":"2024-09-18T01:03:28.564756+0000","last_became_active":"2024-09-18T01:02:41.459321+0000","last_became_peered":"2024-09-18T01:02:41.459321+0000","last_unstale":"2024-09-18T01:03:28.564756+0000","last_undegraded":"2024-09-18T01:03:28.564756+0000","last_fullsized":"2024-09-18T01:03:28.564756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:13.145663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061623500000000002,"stat_sum":{"num_bytes":47937061,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":3888,"num_read_kb":59080,"num_write":331,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"59'511","reported_seq":4215,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.761661+0000","last_change":"2024-09-18T01:02:49.315232+0000","last_active":"2024-09-18T01:03:28.761661+0000","last_peered":"2024-09-18T01:03:28.761661+0000","last_clean":"2024-09-18T01:03:28.761661+0000","last_became_active":"2024-09-18T01:02:49.293095+0000","last_became_peered":"2024-09-18T01:02:49.293095+0000","last_unstale":"2024-09-18T01:03:28.761661+0000","last_undegraded":"2024-09-18T01:03:28.761661+0000","last_fullsized":"2024-09-18T01:03:28.761661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:17.702565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097728599999999995,"stat_sum":{"num_bytes":27739725,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5287,"num_read_kb":17711,"num_write":761,"num_write_kb":2931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"59'219","reported_seq":2142,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.775439+0000","last_change":"2024-09-18T01:03:21.299624+0000","last_active":"2024-09-18T01:03:28.775439+0000","last_peered":"2024-09-18T01:03:28.775439+0000","last_clean":"2024-09-18T01:03:28.775439+0000","last_became_active":"2024-09-18T01:03:21.298726+0000","last_became_peered":"2024-09-18T01:03:21.298726+0000","last_unstale":"2024-09-18T01:03:28.775439+0000","last_undegraded":"2024-09-18T01:03:28.775439+0000","last_fullsized":"2024-09-18T01:03:28.775439+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:26.762981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099035500000000005,"stat_sum":{"num_bytes":14968494,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":2070,"num_read_kb":19451,"num_write":267,"num_write_kb":5646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":3286625,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"59'176","reported_seq":4649,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.913940+0000","last_change":"2024-09-18T01:02:43.464371+0000","last_active":"2024-09-18T01:03:28.913940+0000","last_peered":"2024-09-18T01:03:28.913940+0000","last_clean":"2024-09-18T01:03:28.913940+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:03:28.913940+0000","last_undegraded":"2024-09-18T01:03:28.913940+0000","last_fullsized":"2024-09-18T01:03:28.913940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:20:34.561443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077757900000000003,"stat_sum":{"num_bytes":19914373,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":4443,"num_read_kb":22697,"num_write":160,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"59'376","reported_seq":3560,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.892471+0000","last_change":"2024-09-18T01:02:43.464338+0000","last_active":"2024-09-18T01:03:28.892471+0000","last_peered":"2024-09-18T01:03:28.892471+0000","last_clean":"2024-09-18T01:03:28.892471+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:03:28.892471+0000","last_undegraded":"2024-09-18T01:03:28.892471+0000","last_fullsized":"2024-09-18T01:03:28.892471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:24.155448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080305299999999997,"stat_sum":{"num_bytes":11473500,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":4720,"num_read_kb":5951,"num_write":435,"num_write_kb":1294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479546+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:03:28.479546+0000","last_peered":"2024-09-18T01:03:28.479546+0000","last_clean":"2024-09-18T01:03:28.479546+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:03:28.479546+0000","last_undegraded":"2024-09-18T01:03:28.479546+0000","last_fullsized":"2024-09-18T01:03:28.479546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"59'287","reported_seq":5186,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906478+0000","last_change":"2024-09-18T01:02:43.469252+0000","last_active":"2024-09-18T01:03:28.906478+0000","last_peered":"2024-09-18T01:03:28.906478+0000","last_clean":"2024-09-18T01:03:28.906478+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:03:28.906478+0000","last_undegraded":"2024-09-18T01:03:28.906478+0000","last_fullsized":"2024-09-18T01:03:28.906478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:52.498307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066086499999999996,"stat_sum":{"num_bytes":79278629,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":5452,"num_read_kb":64068,"num_write":374,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.467157+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:03:28.467157+0000","last_peered":"2024-09-18T01:03:28.467157+0000","last_clean":"2024-09-18T01:03:28.467157+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:03:28.467157+0000","last_undegraded":"2024-09-18T01:03:28.467157+0000","last_fullsized":"2024-09-18T01:03:28.467157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"58'1578","reported_seq":7474,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.774811+0000","last_change":"2024-09-18T01:02:43.469995+0000","last_active":"2024-09-18T01:03:28.774811+0000","last_peered":"2024-09-18T01:03:28.774811+0000","last_clean":"2024-09-18T01:03:28.774811+0000","last_became_active":"2024-09-18T01:02:41.456801+0000","last_became_peered":"2024-09-18T01:02:41.456801+0000","last_unstale":"2024-09-18T01:03:28.774811+0000","last_undegraded":"2024-09-18T01:03:28.774811+0000","last_fullsized":"2024-09-18T01:03:28.774811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1578,"log_dups_size":0,"ondisk_log_size":1578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:53:50.463232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159869,"stat_sum":{"num_bytes":25973986,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":14566,"num_read_kb":30805,"num_write":2683,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479533+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:03:28.479533+0000","last_peered":"2024-09-18T01:03:28.479533+0000","last_clean":"2024-09-18T01:03:28.479533+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:03:28.479533+0000","last_undegraded":"2024-09-18T01:03:28.479533+0000","last_fullsized":"2024-09-18T01:03:28.479533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"59'179","reported_seq":3583,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.898386+0000","last_change":"2024-09-18T01:02:43.476334+0000","last_active":"2024-09-18T01:03:28.898386+0000","last_peered":"2024-09-18T01:03:28.898386+0000","last_clean":"2024-09-18T01:03:28.898386+0000","last_became_active":"2024-09-18T01:02:41.457022+0000","last_became_peered":"2024-09-18T01:02:41.457022+0000","last_unstale":"2024-09-18T01:03:28.898386+0000","last_undegraded":"2024-09-18T01:03:28.898386+0000","last_fullsized":"2024-09-18T01:03:28.898386+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:35.329583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050705100000000001,"stat_sum":{"num_bytes":10953812,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":3710,"num_read_kb":6800,"num_write":166,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.467075+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:03:28.467075+0000","last_peered":"2024-09-18T01:03:28.467075+0000","last_clean":"2024-09-18T01:03:28.467075+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:03:28.467075+0000","last_undegraded":"2024-09-18T01:03:28.467075+0000","last_fullsized":"2024-09-18T01:03:28.467075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"59'181","reported_seq":6078,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.899074+0000","last_change":"2024-09-18T01:02:49.294263+0000","last_active":"2024-09-18T01:03:28.899074+0000","last_peered":"2024-09-18T01:03:28.899074+0000","last_clean":"2024-09-18T01:03:28.899074+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:03:28.899074+0000","last_undegraded":"2024-09-18T01:03:28.899074+0000","last_fullsized":"2024-09-18T01:03:28.899074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:55.474675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22912544,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":5881,"num_read_kb":32468,"num_write":204,"num_write_kb":10841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'404","reported_seq":5357,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.900504+0000","last_change":"2024-09-18T01:02:43.473795+0000","last_active":"2024-09-18T01:03:28.900504+0000","last_peered":"2024-09-18T01:03:28.900504+0000","last_clean":"2024-09-18T01:03:28.900504+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:03:28.900504+0000","last_undegraded":"2024-09-18T01:03:28.900504+0000","last_fullsized":"2024-09-18T01:03:28.900504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:09:19.793410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031347599999999998,"stat_sum":{"num_bytes":42097305,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":6750,"num_read_kb":44974,"num_write":449,"num_write_kb":5570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.467245+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:03:28.467245+0000","last_peered":"2024-09-18T01:03:28.467245+0000","last_clean":"2024-09-18T01:03:28.467245+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:03:28.467245+0000","last_undegraded":"2024-09-18T01:03:28.467245+0000","last_fullsized":"2024-09-18T01:03:28.467245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'159","reported_seq":4247,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.773315+0000","last_change":"2024-09-18T01:02:43.476102+0000","last_active":"2024-09-18T01:03:28.773315+0000","last_peered":"2024-09-18T01:03:28.773315+0000","last_clean":"2024-09-18T01:03:28.773315+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:03:28.773315+0000","last_undegraded":"2024-09-18T01:03:28.773315+0000","last_fullsized":"2024-09-18T01:03:28.773315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:01.384338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043811300000000001,"stat_sum":{"num_bytes":35942146,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":4069,"num_read_kb":24616,"num_write":167,"num_write_kb":5011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.481572+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:03:28.481572+0000","last_peered":"2024-09-18T01:03:28.481572+0000","last_clean":"2024-09-18T01:03:28.481572+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:03:28.481572+0000","last_undegraded":"2024-09-18T01:03:28.481572+0000","last_fullsized":"2024-09-18T01:03:28.481572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'242","reported_seq":3457,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.779540+0000","last_change":"2024-09-18T01:02:43.468762+0000","last_active":"2024-09-18T01:03:28.779540+0000","last_peered":"2024-09-18T01:03:28.779540+0000","last_clean":"2024-09-18T01:03:28.779540+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:03:28.779540+0000","last_undegraded":"2024-09-18T01:03:28.779540+0000","last_fullsized":"2024-09-18T01:03:28.779540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:47:06.064710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048296199999999997,"stat_sum":{"num_bytes":22983680,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":4050,"num_read_kb":24657,"num_write":213,"num_write_kb":7184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479228+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:03:28.479228+0000","last_peered":"2024-09-18T01:03:28.479228+0000","last_clean":"2024-09-18T01:03:28.479228+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:03:28.479228+0000","last_undegraded":"2024-09-18T01:03:28.479228+0000","last_fullsized":"2024-09-18T01:03:28.479228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"59'284","reported_seq":3694,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.772934+0000","last_change":"2024-09-18T01:02:49.337150+0000","last_active":"2024-09-18T01:03:28.772934+0000","last_peered":"2024-09-18T01:03:28.772934+0000","last_clean":"2024-09-18T01:03:28.772934+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:03:28.772934+0000","last_undegraded":"2024-09-18T01:03:28.772934+0000","last_fullsized":"2024-09-18T01:03:28.772934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:56:12.155435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049608300000000003,"stat_sum":{"num_bytes":8573077,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":4466,"num_read_kb":22223,"num_write":253,"num_write_kb":4370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":79,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.481696+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:03:28.481696+0000","last_peered":"2024-09-18T01:03:28.481696+0000","last_clean":"2024-09-18T01:03:28.481696+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:03:28.481696+0000","last_undegraded":"2024-09-18T01:03:28.481696+0000","last_fullsized":"2024-09-18T01:03:28.481696+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":253,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.479235+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:03:28.479235+0000","last_peered":"2024-09-18T01:03:28.479235+0000","last_clean":"2024-09-18T01:03:28.479235+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:03:28.479235+0000","last_undegraded":"2024-09-18T01:03:28.479235+0000","last_fullsized":"2024-09-18T01:03:28.479235+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"59'204","reported_seq":4502,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.898145+0000","last_change":"2024-09-18T01:02:43.468997+0000","last_active":"2024-09-18T01:03:28.898145+0000","last_peered":"2024-09-18T01:03:28.898145+0000","last_clean":"2024-09-18T01:03:28.898145+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:03:28.898145+0000","last_undegraded":"2024-09-18T01:03:28.898145+0000","last_fullsized":"2024-09-18T01:03:28.898145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:44:04.566739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063056400000000002,"stat_sum":{"num_bytes":23992937,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":4519,"num_read_kb":16351,"num_write":234,"num_write_kb":8930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"59'247","reported_seq":3659,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.561513+0000","last_change":"2024-09-18T01:02:43.474261+0000","last_active":"2024-09-18T01:03:28.561513+0000","last_peered":"2024-09-18T01:03:28.561513+0000","last_clean":"2024-09-18T01:03:28.561513+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:03:28.561513+0000","last_undegraded":"2024-09-18T01:03:28.561513+0000","last_fullsized":"2024-09-18T01:03:28.561513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:32:50.058657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065863900000000003,"stat_sum":{"num_bytes":15702558,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":3727,"num_read_kb":20808,"num_write":302,"num_write_kb":9105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"59'531","reported_seq":3628,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.557837+0000","last_change":"2024-09-18T01:02:43.464113+0000","last_active":"2024-09-18T01:03:28.557837+0000","last_peered":"2024-09-18T01:03:28.557837+0000","last_clean":"2024-09-18T01:03:28.557837+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:03:28.557837+0000","last_undegraded":"2024-09-18T01:03:28.557837+0000","last_fullsized":"2024-09-18T01:03:28.557837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:56.079458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066269099999999997,"stat_sum":{"num_bytes":13186186,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":5140,"num_read_kb":12829,"num_write":711,"num_write_kb":2229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"59'319","reported_seq":4177,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906375+0000","last_change":"2024-09-18T01:02:43.476485+0000","last_active":"2024-09-18T01:03:28.906375+0000","last_peered":"2024-09-18T01:03:28.906375+0000","last_clean":"2024-09-18T01:03:28.906375+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:03:28.906375+0000","last_undegraded":"2024-09-18T01:03:28.906375+0000","last_fullsized":"2024-09-18T01:03:28.906375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:50:26.218489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054801899999999996,"stat_sum":{"num_bytes":22089760,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":5374,"num_read_kb":28757,"num_write":268,"num_write_kb":5624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"59'161","reported_seq":3184,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.793063+0000","last_change":"2024-09-18T01:02:49.302821+0000","last_active":"2024-09-18T01:03:28.793063+0000","last_peered":"2024-09-18T01:03:28.793063+0000","last_clean":"2024-09-18T01:03:28.793063+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:03:28.793063+0000","last_undegraded":"2024-09-18T01:03:28.793063+0000","last_fullsized":"2024-09-18T01:03:28.793063+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:45.502789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043103,"stat_sum":{"num_bytes":14301184,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":2929,"num_read_kb":17165,"num_write":158,"num_write_kb":3822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"59'241","reported_seq":3891,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.776643+0000","last_change":"2024-09-18T01:02:43.464136+0000","last_active":"2024-09-18T01:03:28.776643+0000","last_peered":"2024-09-18T01:03:28.776643+0000","last_clean":"2024-09-18T01:03:28.776643+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:03:28.776643+0000","last_undegraded":"2024-09-18T01:03:28.776643+0000","last_fullsized":"2024-09-18T01:03:28.776643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:55.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061418400000000004,"stat_sum":{"num_bytes":13560832,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":4352,"num_read_kb":13338,"num_write":218,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"59'189","reported_seq":4731,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.911247+0000","last_change":"2024-09-18T01:02:49.312596+0000","last_active":"2024-09-18T01:03:28.911247+0000","last_peered":"2024-09-18T01:03:28.911247+0000","last_clean":"2024-09-18T01:03:28.911247+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:03:28.911247+0000","last_undegraded":"2024-09-18T01:03:28.911247+0000","last_fullsized":"2024-09-18T01:03:28.911247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:02:40.082959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013236470000000001,"stat_sum":{"num_bytes":30078513,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":4593,"num_read_kb":23323,"num_write":168,"num_write_kb":6694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"59'382","reported_seq":11200,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.785863+0000","last_change":"2024-09-18T01:02:49.321182+0000","last_active":"2024-09-18T01:03:28.785863+0000","last_peered":"2024-09-18T01:03:28.785863+0000","last_clean":"2024-09-18T01:03:28.785863+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:03:28.785863+0000","last_undegraded":"2024-09-18T01:03:28.785863+0000","last_fullsized":"2024-09-18T01:03:28.785863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:56.098763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011817170000000001,"stat_sum":{"num_bytes":8214543,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":9347,"num_read_kb":15854,"num_write":240,"num_write_kb":3235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"59'187","reported_seq":2194,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.899472+0000","last_change":"2024-09-18T01:02:49.345428+0000","last_active":"2024-09-18T01:03:28.899472+0000","last_peered":"2024-09-18T01:03:28.899472+0000","last_clean":"2024-09-18T01:03:28.899472+0000","last_became_active":"2024-09-18T01:02:49.296123+0000","last_became_peered":"2024-09-18T01:02:49.296123+0000","last_unstale":"2024-09-18T01:03:28.899472+0000","last_undegraded":"2024-09-18T01:03:28.899472+0000","last_fullsized":"2024-09-18T01:03:28.899472+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:33.113422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053670300000000003,"stat_sum":{"num_bytes":17298696,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":2447,"num_read_kb":14495,"num_write":167,"num_write_kb":2630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"59'177","reported_seq":2704,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.897886+0000","last_change":"2024-09-18T01:02:43.463798+0000","last_active":"2024-09-18T01:03:28.897886+0000","last_peered":"2024-09-18T01:03:28.897886+0000","last_clean":"2024-09-18T01:03:28.897886+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:03:28.897886+0000","last_undegraded":"2024-09-18T01:03:28.897886+0000","last_fullsized":"2024-09-18T01:03:28.897886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:49.447708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042487699999999999,"stat_sum":{"num_bytes":7055872,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":2884,"num_read_kb":10827,"num_write":161,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"59'455","reported_seq":7706,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.679660+0000","last_change":"2024-09-18T01:02:43.468742+0000","last_active":"2024-09-18T01:03:28.679660+0000","last_peered":"2024-09-18T01:03:28.679660+0000","last_clean":"2024-09-18T01:03:28.679660+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:03:28.679660+0000","last_undegraded":"2024-09-18T01:03:28.679660+0000","last_fullsized":"2024-09-18T01:03:28.679660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:25:48.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043475499999999999,"stat_sum":{"num_bytes":38362802,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":9296,"num_read_kb":31449,"num_write":545,"num_write_kb":12457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"59'165","reported_seq":3165,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T01:03:28.906375+0000","last_change":"2024-09-18T01:02:49.356374+0000","last_active":"2024-09-18T01:03:28.906375+0000","last_peered":"2024-09-18T01:03:28.906375+0000","last_clean":"2024-09-18T01:03:28.906375+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:03:28.906375+0000","last_undegraded":"2024-09-18T01:03:28.906375+0000","last_fullsized":"2024-09-18T01:03:28.906375+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:22.333692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311600000000002,"stat_sum":{"num_bytes":19833872,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":2938,"num_read_kb":11227,"num_write":165,"num_write_kb":5592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":734984338,"num_objects":865,"num_object_clones":170,"num_object_copies":1730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":865,"num_whiteouts":323,"num_read":157238,"num_read_kb":723932,"num_write":11931,"num_write_kb":181564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":3303066,"num_keys_recovered":14,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28958720,"data_stored":237762852,"data_compressed":5582956,"data_compressed_allocated":19038208,"data_compressed_original":230079488,"omap_allocated":0,"internal_metadata":0},"log_size":10073,"ondisk_log_size":10073,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640271,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640273,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49392,"kb_used_data":5728,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94322448,"statfs":{"total":96636764160,"available":96586186752,"internally_reserved":0,"allocated":5865472,"data_stored":42484418,"data_compressed":1007932,"data_compressed_allocated":3436544,"data_compressed_original":40852992,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640272,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45288,"kb_used_data":5784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326552,"statfs":{"total":96636764160,"available":96590389248,"internally_reserved":0,"allocated":5922816,"data_stored":41846576,"data_compressed":1042185,"data_compressed_allocated":3325952,"data_compressed_original":40109056,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640271,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62860,"kb_used_data":5304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308980,"statfs":{"total":96636764160,"available":96572395520,"internally_reserved":0,"allocated":5431296,"data_stored":42227394,"data_compressed":751685,"data_compressed_allocated":3158016,"data_compressed_original":40742912,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640273,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51996,"kb_used_data":6896,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319844,"statfs":{"total":96636764160,"available":96583520256,"internally_reserved":0,"allocated":7061504,"data_stored":54209882,"data_compressed":1312067,"data_compressed_allocated":4354048,"data_compressed_original":52379136,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34804,"kb_used_data":7780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337036,"statfs":{"total":96636764160,"available":96601124864,"internally_reserved":0,"allocated":7966720,"data_stored":60291330,"data_compressed":1506578,"data_compressed_allocated":4878336,"data_compressed_original":58108928,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7229440,"data_stored":59433661,"data_compressed":1498028,"data_compressed_allocated":4841472,"data_compressed_original":57535488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6434816,"data_stored":53007669,"data_compressed":1291140,"data_compressed_allocated":4280320,"data_compressed_original":51346944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4829184,"data_stored":42025498,"data_compressed":751819,"data_compressed_allocated":3166208,"data_compressed_original":40759296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5275648,"data_stored":41078888,"data_compressed":1034037,"data_compressed_allocated":3313664,"data_compressed_original":39584768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5189632,"data_stored":42217136,"data_compressed":1007932,"data_compressed_allocated":3436544,"data_compressed_original":40852992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:03:33.304 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:03:33.304 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:33.304 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T01:03:33.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:03:33.305 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:33.297+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:33.399 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:33.398+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:33.407 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:33.408 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:33.408 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:33.408 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:33.416 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:33.500 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:33.498+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:33.522 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:33.522 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:33.522 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:33.522 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:33.523 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:33.531 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:33.601 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:33.600+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:33.605 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 64, 2024-09-18T01:03:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-18T01:02:13.493841+0000", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-18T01:03:33.491513+0000", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-18T01:02:24.290620+0000", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-18T01:02:48.274416+0000", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-18T01:03:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-18T01:03:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:24.388064+0000", 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:30.676199+0000", 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2699999809265137, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2699999809265137, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T01:03:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:40.390732+0000", 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "64", 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 24, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 64, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.5099999904632568, 2024-09-18T01:03:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.5099999904632568, 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.25, 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.25, 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4563fb16-8745-4e75-b654-1e96403d91b9", 2024-09-18T01:03:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-18T01:03:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-18T01:03:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/1626166637", 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/1626166637", 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/1626166637", 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/1626166637", 2024-09-18T01:03:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "634965cb-d444-4721-b34a-afbc49105cab", 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-18T01:03:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/2210970259", 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/2210970259", 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/2210970259", 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/2210970259", 2024-09-18T01:03:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "83443eab-2fa4-4c7e-a459-edb1bb0ed8d0", 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/617133675", 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/617133675", 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/617133675", 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/617133675", 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "5797982a-71c1-4d6a-9604-00fbc6bd16d9", 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 53, 2024-09-18T01:03:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6808", 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6809", 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6810", 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6811", 2024-09-18T01:03:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6814", 2024-09-18T01:03:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6815", 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6812", 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6813", 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6809/151031464", 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6811/151031464", 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6815/151031464", 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6813/151031464", 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f79e3329-5481-4f33-8d28-6386af186492", 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 28, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6816", 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6817", 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6818", 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6819", 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6822", 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6823", 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6820", 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6821", 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6817/1499036965", 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6819/1499036965", 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6823/1499036965", 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6821/1499036965", 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "8c61d2ec-b111-4783-a016-336ed33a0694", 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 20, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6800", 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6801", 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6802", 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6803", 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6806", 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6807", 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6804", 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6805", 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6801/2310498524", 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6803/2310498524", 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6807/2310498524", 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6805/2310498524", 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.877700+0000", 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.360930+0000", 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.403205+0000", 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.952123+0000", 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:22.099545+0000", 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:33.668 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.694132+0000", 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.16", 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: 3, 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: 0 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.669 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-18T01:03:33.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-18T01:03:33.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:33.672 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:33.673 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone21 from 0xdccb800 (overlap 0x604e000) to 0x39c8600 2024-09-18T01:03:33.675 INFO:tasks.thrashosds.thrasher:j is {'epoch': 64, 'fsid': '62b46b83-8ef2-466f-b28b-2d97f265422e', 'created': '2024-09-18T01:02:13.493841+0000', 'modified': '2024-09-18T01:03:33.491513+0000', 'last_up_change': '2024-09-18T01:02:24.290620+0000', 'last_in_change': '2024-09-18T01:02:48.274416+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:02:24.388064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:02:30.676199+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:02:40.390732+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4563fb16-8745-4e75-b654-1e96403d91b9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 1626166637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 1626166637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 1626166637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 1626166637}]}, 'public_addr': '172.21.15.31:6817/1626166637', 'cluster_addr': '172.21.15.31:6819/1626166637', 'heartbeat_back_addr': '172.21.15.31:6823/1626166637', 'heartbeat_front_addr': '172.21.15.31:6821/1626166637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '634965cb-d444-4721-b34a-afbc49105cab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 2210970259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 2210970259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 2210970259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 2210970259}]}, 'public_addr': '172.21.15.31:6801/2210970259', 'cluster_addr': '172.21.15.31:6803/2210970259', 'heartbeat_back_addr': '172.21.15.31:6807/2210970259', 'heartbeat_front_addr': '172.21.15.31:6805/2210970259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 617133675}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 617133675}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 617133675}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 617133675}]}, 'public_addr': '172.21.15.31:6809/617133675', 'cluster_addr': '172.21.15.31:6811/617133675', 'heartbeat_back_addr': '172.21.15.31:6815/617133675', 'heartbeat_front_addr': '172.21.15.31:6813/617133675', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6808', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6809', 'nonce': 151031464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6810', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6811', 'nonce': 151031464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6814', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6815', 'nonce': 151031464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6812', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6813', 'nonce': 151031464}]}, 'public_addr': '172.21.15.98:6809/151031464', 'cluster_addr': '172.21.15.98:6811/151031464', 'heartbeat_back_addr': '172.21.15.98:6815/151031464', 'heartbeat_front_addr': '172.21.15.98:6813/151031464', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f79e3329-5481-4f33-8d28-6386af186492', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6816', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6817', 'nonce': 1499036965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6818', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6819', 'nonce': 1499036965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6822', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6823', 'nonce': 1499036965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6820', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6821', 'nonce': 1499036965}]}, 'public_addr': '172.21.15.98:6817/1499036965', 'cluster_addr': '172.21.15.98:6819/1499036965', 'heartbeat_back_addr': '172.21.15.98:6823/1499036965', 'heartbeat_front_addr': '172.21.15.98:6821/1499036965', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8c61d2ec-b111-4783-a016-336ed33a0694', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6800', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6801', 'nonce': 2310498524}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6802', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6803', 'nonce': 2310498524}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6806', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6807', 'nonce': 2310498524}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6804', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6805', 'nonce': 2310498524}]}, 'public_addr': '172.21.15.98:6801/2310498524', 'cluster_addr': '172.21.15.98:6803/2310498524', 'heartbeat_back_addr': '172.21.15.98:6807/2310498524', 'heartbeat_front_addr': '172.21.15.98:6805/2310498524', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.877700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.360930+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.403205+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.952123+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:22.099545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.694132+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.16', 'osds': [3, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:03:33.675 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.16 2024-09-18T01:03:33.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.16 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.595699590999999, 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.676 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:03:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:03:33.680 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.59568844, 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.681 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:03:33.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:03:33.685 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.595676610000002, 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.686 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:03:33.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.690 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.595659927, 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:33.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:03:33.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.595648690000001, 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:03:33.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:03:33.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:03:33.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.700 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.594326585000001, 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:03:33.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.705 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.594313870000001, 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:03:33.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.594268886000002, 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.710 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:03:33.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:03:33.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.594241262000001, 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.715 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:03:33.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:03:33.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.590693883, 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:03:33.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:03:33.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:33.724 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.590333776000001, 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:03:33.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:03:33.729 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.729 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.729 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.729 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.729 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.590307663000001, 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:03:33.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23940 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894162912] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.58813795, 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037903539999999999, 2024-09-18T01:03:33.738 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23940 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051178+0000", 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1299999999999999e-07 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051180+0000", 2024-09-18T01:03:33.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4190000000000001e-06 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051183+0000", 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.774e-06 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051185+0000", 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5330000000000001e-06 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.088984+0000", 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037798867999999999 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089016+0000", 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2333999999999999e-05 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089082+0000", 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5399000000000002e-05 2024-09-18T01:03:33.742 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.588128274999999, 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.743 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.588118479999999, 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:03:33.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:03:33.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.588107700999998, 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:03:33.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.757 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.588087904999998, 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:03:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.586304384999998, 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.762 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:03:33.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:03:33.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.586277547000002, 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:03:33.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:03:33.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 31.586266998999999, 2024-09-18T01:03:33.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:03:33.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.775 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.776 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.776 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:33.776 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:33.776 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:33.776 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:33.778 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:33.775+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:33.878 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:33.877+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:33.880 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:33.881 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:33.890 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:33.980 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:33.978+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:34.034 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:34.034 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:34.034 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:34.034 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:34.034 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:34.043 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:34.081 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:34.079+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:34.146 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848947719000002, 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.147 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.148 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:03:34.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848923601999999, 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:03:34.151 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:03:34.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848844362000001, 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:03:34.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848808142999999, 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.160 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:34.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848779530000002, 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:03:34.164 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:03:34.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848758664999998, 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.169 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:03:34.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.172 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848140948000001, 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.173 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.174 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:03:34.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.174 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:03:34.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848111680999999, 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.178 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.179 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.180 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:03:34.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.848045389999999, 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:03:34.182 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.183 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:03:34.184 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.186 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847924623000001, 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.187 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:34.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:03:34.190 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847706176999999, 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.192 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.193 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:03:34.194 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847684816000001, 2024-09-18T01:03:34.196 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:34.197 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.198 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.198 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.198 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:34.198 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.198 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:03:34.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.200 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847530877000001, 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.201 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.202 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.203 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:03:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847511159, 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:03:34.205 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.206 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:03:34.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:03:34.208 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:34.209 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847490849, 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.210 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:03:34.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.213 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.214 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847470962999999, 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.215 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:03:34.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.216 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.216 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.216 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.216 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:34.216 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:34.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:03:34.218 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.219 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847451004, 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:03:34.220 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:34.221 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.222 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:03:34.223 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.224 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847431006000001, 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:03:34.225 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.226 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.227 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.228 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847411339000001, 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.229 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.230 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:03:34.231 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:03:34.232 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 27.847391436999999, 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.233 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.234 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:03:34.235 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.236 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.237 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:34.239 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:34.239 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:34.237+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:34.330 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-18T01:03:34.339 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:34.337+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:34.385 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:34.385 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:34.385 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:34.394 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:34.397 INFO:teuthology.orchestra.run.smithi164.stdout:2925 read 0x9559400 thru 0x9564ec4 (0xbac5 bytes) 2024-09-18T01:03:34.404 INFO:teuthology.orchestra.run.smithi164.stdout:2926 trunc from 0xdccb800 to 0x9516000 2024-09-18T01:03:34.440 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:34.439+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:34.447 INFO:teuthology.orchestra.run.smithi164.stdout:2929 write 0xb661c00 thru 0xb666567 (0x4968 bytes) 2024-09-18T01:03:34.448 INFO:teuthology.orchestra.run.smithi164.stdout:2932 write 0xa3e2800 thru 0xa3e877d (0x5f7e bytes) 2024-09-18T01:03:34.448 INFO:teuthology.orchestra.run.smithi164.stdout:2933 write 0xa7bfc00 thru 0xa7cde79 (0xe27a bytes) 2024-09-18T01:03:34.448 INFO:teuthology.orchestra.run.smithi164.stdout:2937 read 0x7e32c00 thru 0x7e3f428 (0xc829 bytes) 2024-09-18T01:03:34.449 INFO:teuthology.orchestra.run.smithi164.stdout:2939 read 0x70c1200 thru 0x70cbebc (0xacbd bytes) 2024-09-18T01:03:34.451 INFO:teuthology.orchestra.run.smithi164.stdout:2941 trunc from 0xb666600 to 0xa0ad600 2024-09-18T01:03:34.479 INFO:teuthology.orchestra.run.smithi164.stdout:2942 punch from 0x7275800 to 0x7278800, (0x3000 bytes) 2024-09-18T01:03:34.495 INFO:teuthology.orchestra.run.smithi164.stdout:2943 write 0xdddac00 thru 0xdde74c0 (0xc8c1 bytes) 2024-09-18T01:03:34.497 INFO:teuthology.orchestra.run.smithi164.stdout:2948 trunc from 0xdde7600 to 0xc56a000 2024-09-18T01:03:34.504 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:34.505 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:34.505 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:34.505 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:34.505 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:34.506 INFO:teuthology.orchestra.run.smithi031.stderr:clear 3.16 pg_upmap mapping 2024-09-18T01:03:34.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:03:34.514 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:34.524 INFO:teuthology.orchestra.run.smithi164.stdout:2949 write 0x4b28600 thru 0x4b37464 (0xee65 bytes) 2024-09-18T01:03:34.541 INFO:teuthology.orchestra.run.smithi164.stdout:2951 trunc from 0xc56a000 to 0x6c600 2024-09-18T01:03:34.555 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:34.554+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:34.643 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:34.641+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:34.660 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:34.660 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:34.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:34.660 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 62.284211237999997, 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:34.661 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:34.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:34.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.667 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 61.286199932999999, 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:34.668 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:34.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:34.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:34.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.278522981000002, 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.675 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:34.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:34.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.180665030999997, 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.682 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:34.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:34.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:34.688 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.324584019, 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.689 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:34.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:34.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:34.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:34.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:34.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.695 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.324214722999997, 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.696 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:34.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:34.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.322475877999999, 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:34.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:34.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.322403987999998, 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:34.706 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:34.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.710 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.234537977999999, 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:34.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.716 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.228346719000001, 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:34.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.721 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.226817034, 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:34.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:34.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.226437345000001, 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:34.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:34.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.733 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.210938941999999, 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:34.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:34.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:34.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.210732182999998, 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.740 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:34.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:34.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.210314425999997, 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:34.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.210105188, 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:34.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.210041253999997, 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.755 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:34.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.759 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 47.209293049000003, 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:34.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:34.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 46.950354244000003, 2024-09-18T01:03:34.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:34.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:34.770 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 46.949490625999999, 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:34.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:34.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:34.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:34.776 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:34.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:34.778 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:34.777+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:34.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:34.878 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:34.876+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:34.887 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:34.887 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:34.887 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:34.887 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:34.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:34.901 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:34.902 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:34.919 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:34.936 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:34.953 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:34.953 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:34.969 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:34.970 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:34.979 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:34.978+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:34.987 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:35.010 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:35.010 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:35.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:35.080 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:35.079+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:35.165 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:35.165 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:35.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:35.219 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:35.217+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:35.282 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:35.330 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:35.331 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:35.281+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:35.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:35.416 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:35.414+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:35.480 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:35.481 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:35.484 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:35.482+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:35.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:03:35.584 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:35.582+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:35.611 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:35.611 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:35.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:35.704 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:35.702+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:35.768 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:35.768 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:35.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:35.786 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:35.785+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:35.887 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:35.885+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:35.924 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:35.924 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:35.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:36.019 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:36.017+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:36.082 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:36.082 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:36.089 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:36.088+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:36.091 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:36.190 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:36.189+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:36.206 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:36.206 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:36.215 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:36.292 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:36.290+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:36.359 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:36.359 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:36.368 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:36.412 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:36.411+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:36.470 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:36.471 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:36.479 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:36.520 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:36.519+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:36.536 INFO:teuthology.orchestra.run.smithi164.stdout:2952 punch from 0x5e800 to 0x66200, (0x7a00 bytes) 2024-09-18T01:03:36.542 INFO:teuthology.orchestra.run.smithi164.stdout:2954 trunc from 0x6c600 to 0x16cfc00 2024-09-18T01:03:36.559 INFO:teuthology.orchestra.run.smithi164.stdout:2955 trunc from 0x16cfc00 to 0x4dddc00 2024-09-18T01:03:36.569 INFO:teuthology.orchestra.run.smithi164.stdout:2957 punch from 0x4504800 to 0x4512000, (0xd800 bytes) 2024-09-18T01:03:36.570 INFO:teuthology.orchestra.run.smithi164.stdout:2958 read 0x476f400 thru 0x476faf8 (0x6f9 bytes) 2024-09-18T01:03:36.573 INFO:teuthology.orchestra.run.smithi164.stdout:2959 trunc from 0x4dddc00 to 0x6422400 2024-09-18T01:03:36.587 INFO:teuthology.orchestra.run.smithi164.stdout:2960 punch from 0x5456c00 to 0x5461600, (0xaa00 bytes) 2024-09-18T01:03:36.589 INFO:teuthology.orchestra.run.smithi164.stdout:2961 read 0x3c70c00 thru 0x3c7f922 (0xed23 bytes) 2024-09-18T01:03:36.594 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:36.593+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:36.598 INFO:teuthology.orchestra.run.smithi164.stdout:2962 trunc from 0x6422400 to 0xb78e00 2024-09-18T01:03:36.632 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:36.632 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:36.638 INFO:teuthology.orchestra.run.smithi164.stdout:2963 write 0x99b9e00 thru 0x99c5bb2 (0xbdb3 bytes) 2024-09-18T01:03:36.639 INFO:teuthology.orchestra.run.smithi164.stdout:2964 read 0x659d200 thru 0x65ad0a7 (0xfea8 bytes) 2024-09-18T01:03:36.639 INFO:teuthology.orchestra.run.smithi164.stdout:2965 punch from 0x27f9e00 to 0x2803a00, (0x9c00 bytes) 2024-09-18T01:03:36.640 INFO:teuthology.orchestra.run.smithi164.stdout:2966 read 0x782600 thru 0x78e7f4 (0xc1f5 bytes) 2024-09-18T01:03:36.640 INFO:teuthology.orchestra.run.smithi164.stdout:2967 read 0x76d8200 thru 0x76d8911 (0x712 bytes) 2024-09-18T01:03:36.640 INFO:teuthology.orchestra.run.smithi164.stdout:2969 read 0x6804000 thru 0x6810618 (0xc619 bytes) 2024-09-18T01:03:36.642 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:36.654 INFO:teuthology.orchestra.run.smithi164.stdout:2971 write 0xb549800 thru 0xb54b54a (0x1d4b bytes) 2024-09-18T01:03:36.655 INFO:teuthology.orchestra.run.smithi164.stdout:2972 read 0x2f3aa00 thru 0x2f4854a (0xdb4b bytes) 2024-09-18T01:03:36.656 INFO:teuthology.orchestra.run.smithi164.stdout:2973 trunc from 0xb54b600 to 0xc56fe00 2024-09-18T01:03:36.669 INFO:teuthology.orchestra.run.smithi164.stdout:2975 read 0x677ce00 thru 0x6785424 (0x8625 bytes) 2024-09-18T01:03:36.669 INFO:teuthology.orchestra.run.smithi164.stdout:2976 read 0xc4f3200 thru 0xc4f7e10 (0x4c11 bytes) 2024-09-18T01:03:36.669 INFO:teuthology.orchestra.run.smithi164.stdout:2977 punch from 0x748e600 to 0x7496a00, (0x8400 bytes) 2024-09-18T01:03:36.672 INFO:teuthology.orchestra.run.smithi164.stdout:2978 read 0x27bc400 thru 0x27c5b75 (0x9776 bytes) 2024-09-18T01:03:36.672 INFO:teuthology.orchestra.run.smithi164.stdout:2979 write 0x24b1200 thru 0x24bc58c (0xb38d bytes) 2024-09-18T01:03:36.683 INFO:teuthology.orchestra.run.smithi164.stdout:2980 trunc from 0xc56fe00 to 0x3d2f600 2024-09-18T01:03:36.695 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:36.693+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:36.702 INFO:teuthology.orchestra.run.smithi164.stdout:2982 read 0x3623400 thru 0x36288b1 (0x54b2 bytes) 2024-09-18T01:03:36.709 INFO:teuthology.orchestra.run.smithi164.stdout:2983 trunc from 0x3d2f600 to 0x9181400 2024-09-18T01:03:36.734 INFO:teuthology.orchestra.run.smithi164.stdout:2984 write 0xddb1000 thru 0xddb801a (0x701b bytes) 2024-09-18T01:03:36.735 INFO:teuthology.orchestra.run.smithi164.stdout:2986 read 0xbd3a800 thru 0xbd3fc04 (0x5405 bytes) 2024-09-18T01:03:36.735 INFO:teuthology.orchestra.run.smithi164.stdout:2988 write 0x89dea00 thru 0x89e1cff (0x3300 bytes) 2024-09-18T01:03:36.739 INFO:teuthology.orchestra.run.smithi164.stdout:2989 write 0x5235200 thru 0x523861e (0x341f bytes) 2024-09-18T01:03:36.739 INFO:teuthology.orchestra.run.smithi164.stdout:2990 read 0x3540600 thru 0x35475b8 (0x6fb9 bytes) 2024-09-18T01:03:36.739 INFO:teuthology.orchestra.run.smithi164.stdout:2991 read 0x4ce4e00 thru 0x4cedf3e (0x913f bytes) 2024-09-18T01:03:36.739 INFO:teuthology.orchestra.run.smithi164.stdout:2993 write 0x6b1c000 thru 0x6b25fce (0x9fcf bytes) 2024-09-18T01:03:36.744 INFO:teuthology.orchestra.run.smithi164.stdout:2994 trunc from 0xddb8200 to 0x6aa6800 2024-09-18T01:03:36.768 INFO:teuthology.orchestra.run.smithi164.stdout:2995 write 0x309c800 thru 0x30a96f6 (0xcef7 bytes) 2024-09-18T01:03:36.769 INFO:teuthology.orchestra.run.smithi164.stdout:2997 read 0x54fce00 thru 0x55010b6 (0x42b7 bytes) 2024-09-18T01:03:36.769 INFO:teuthology.orchestra.run.smithi164.stdout:2999 read 0x382f200 thru 0x3833f9c (0x4d9d bytes) 2024-09-18T01:03:36.773 INFO:teuthology.orchestra.run.smithi164.stdout:3002 read 0x6794200 thru 0x679614d (0x1f4e bytes) 2024-09-18T01:03:36.773 INFO:teuthology.orchestra.run.smithi164.stdout:3003 trunc from 0x6aa6800 to 0x84eb600 2024-09-18T01:03:36.789 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:36.789 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:36.795 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:36.793+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:36.797 INFO:teuthology.orchestra.run.smithi164.stdout:3004 write 0xd02a200 thru 0xd033db5 (0x9bb6 bytes) 2024-09-18T01:03:36.798 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:36.799 INFO:teuthology.orchestra.run.smithi164.stdout:3005 read 0x37c0800 thru 0x37c8b53 (0x8354 bytes) 2024-09-18T01:03:36.799 INFO:teuthology.orchestra.run.smithi164.stdout:3006 read 0x6150000 thru 0x6151505 (0x1506 bytes) 2024-09-18T01:03:36.802 INFO:teuthology.orchestra.run.smithi164.stdout:3007 trunc from 0xd033e00 to 0x984ce00 2024-09-18T01:03:36.820 INFO:teuthology.orchestra.run.smithi164.stdout:3009 read 0x826ce00 thru 0x8274e02 (0x8003 bytes) 2024-09-18T01:03:36.824 INFO:teuthology.orchestra.run.smithi164.stdout:3011 trunc from 0x984ce00 to 0xa372000 2024-09-18T01:03:36.829 INFO:teuthology.orchestra.run.smithi164.stdout:3013 read 0x7bdea00 thru 0x7beb4b3 (0xcab4 bytes) 2024-09-18T01:03:36.834 INFO:teuthology.orchestra.run.smithi164.stdout:3014 trunc from 0xa372000 to 0xddf6e00 2024-09-18T01:03:36.843 INFO:teuthology.orchestra.run.smithi164.stdout:3016 read 0xb2d7600 thru 0xb2e0ec7 (0x98c8 bytes) 2024-09-18T01:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout:3017 write 0xcd76800 thru 0xcd7a37a (0x3b7b bytes) 2024-09-18T01:03:36.844 INFO:teuthology.orchestra.run.smithi164.stdout:3020 punch from 0x1c4d800 to 0x1c4e200, (0xa00 bytes) 2024-09-18T01:03:36.847 INFO:teuthology.orchestra.run.smithi164.stdout:3021 punch from 0xb4e600 to 0xb5a400, (0xbe00 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3022 write 0xb99aa00 thru 0xb99b70f (0xd10 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3024 punch from 0xa994200 to 0xa996e00, (0x2c00 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3025 punch from 0xa677c00 to 0xa682a00, (0xae00 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3029 write 0x4af6000 thru 0x4b032fb (0xd2fc bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3030 read 0x8e3ae00 thru 0x8e46fae (0xc1af bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3032 read 0x6e6d600 thru 0x6e6f9eb (0x23ec bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3034 write 0x425bc00 thru 0x42624ae (0x68af bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3035 read 0x6f00c00 thru 0x6f03ad1 (0x2ed2 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3037 punch from 0x137e800 to 0x138a600, (0xbe00 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3039 read 0x48e3e00 thru 0x48e410a (0x30b bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3040 write 0x3aeb400 thru 0x3af0e88 (0x5a89 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3041 punch from 0x7619800 to 0x7622600, (0x8e00 bytes) 2024-09-18T01:03:36.848 INFO:teuthology.orchestra.run.smithi164.stdout:3042 punch from 0x13c0e00 to 0x13cb000, (0xa200 bytes) 2024-09-18T01:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout:3044 punch from 0x256d800 to 0x257d200, (0xfa00 bytes) 2024-09-18T01:03:36.849 INFO:teuthology.orchestra.run.smithi164.stdout:3045 write 0xb686000 thru 0xb693495 (0xd496 bytes) 2024-09-18T01:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout:3046 punch from 0x4815400 to 0x481be00, (0x6a00 bytes) 2024-09-18T01:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout:3051 write 0x1ce8400 thru 0x1ce96ab (0x12ac bytes) 2024-09-18T01:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout:3054 write 0x87c9e00 thru 0x87d811b (0xe31c bytes) 2024-09-18T01:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout:3056 read 0x8d1de00 thru 0x8d245b1 (0x67b2 bytes) 2024-09-18T01:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout:3057 read 0xd11a00 thru 0xd14f06 (0x3507 bytes) 2024-09-18T01:03:36.850 INFO:teuthology.orchestra.run.smithi164.stdout:3058 trunc from 0xddf6e00 to 0xdecba00 2024-09-18T01:03:36.883 INFO:teuthology.orchestra.run.smithi164.stdout:3059 read 0x823aa00 thru 0x823d0ee (0x26ef bytes) 2024-09-18T01:03:36.888 INFO:teuthology.orchestra.run.smithi164.stdout:3060 trunc from 0xdecba00 to 0xdb75400 2024-09-18T01:03:36.894 INFO:teuthology.orchestra.run.smithi164.stdout:3061 write 0x9c6c400 thru 0x9c76c80 (0xa881 bytes) 2024-09-18T01:03:36.896 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:36.895+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout:3063 read 0x6a57c00 thru 0x6a5ffc0 (0x83c1 bytes) 2024-09-18T01:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout:3065 read 0x4516e00 thru 0x451e5fc (0x77fd bytes) 2024-09-18T01:03:36.901 INFO:teuthology.orchestra.run.smithi164.stdout:3066 write 0x4712000 thru 0x471ea3e (0xca3f bytes) 2024-09-18T01:03:36.907 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:36.907 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:36.915 INFO:teuthology.orchestra.run.smithi164.stdout:3067 trunc from 0xdb75400 to 0x276b400 2024-09-18T01:03:36.917 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:36.950 INFO:teuthology.orchestra.run.smithi164.stdout:3069 read 0x561a00 thru 0x56ede3 (0xd3e4 bytes) 2024-09-18T01:03:36.966 INFO:teuthology.orchestra.run.smithi164.stdout:3072 write 0x4190600 thru 0x4198601 (0x8002 bytes) 2024-09-18T01:03:36.997 INFO:teuthology.orchestra.run.smithi164.stdout:3074 write 0xe751000 thru 0xe752266 (0x1267 bytes) 2024-09-18T01:03:36.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:37.001 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:37.000+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:37.002 INFO:teuthology.orchestra.run.smithi164.stdout:3075 read 0x6c21000 thru 0x6c2b430 (0xa431 bytes) 2024-09-18T01:03:37.002 INFO:teuthology.orchestra.run.smithi164.stdout:3076 punch from 0x242f400 to 0x2435c00, (0x6800 bytes) 2024-09-18T01:03:37.016 INFO:teuthology.orchestra.run.smithi164.stdout:3077 trunc from 0xe752400 to 0x3089200 2024-09-18T01:03:37.045 INFO:teuthology.orchestra.run.smithi164.stdout:3078 read 0x29f3000 thru 0x2a025ad (0xf5ae bytes) 2024-09-18T01:03:37.064 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:37.064 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:37.072 INFO:teuthology.orchestra.run.smithi164.stdout:3079 write 0xd5ec600 thru 0xd5ee66e (0x206f bytes) 2024-09-18T01:03:37.073 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:37.078 INFO:teuthology.orchestra.run.smithi164.stdout:3081 write 0x4c45a00 thru 0x4c4b129 (0x572a bytes) 2024-09-18T01:03:37.078 INFO:teuthology.orchestra.run.smithi164.stdout:3082 trunc from 0xd5ee800 to 0xdbbc400 2024-09-18T01:03:37.094 INFO:teuthology.orchestra.run.smithi164.stdout:3084 punch from 0x325f400 to 0x3269800, (0xa400 bytes) 2024-09-18T01:03:37.098 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:37.096+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:37.110 INFO:teuthology.orchestra.run.smithi164.stdout:3086 trunc from 0xdbbc400 to 0x5041c00 2024-09-18T01:03:37.132 INFO:teuthology.orchestra.run.smithi164.stdout:3087 read 0x364cc00 thru 0x365a3a0 (0xd7a1 bytes) 2024-09-18T01:03:37.132 INFO:teuthology.orchestra.run.smithi164.stdout:3088 read 0x446000 thru 0x44c14d (0x614e bytes) 2024-09-18T01:03:37.132 INFO:teuthology.orchestra.run.smithi164.stdout:3091 write 0x1f9c00 thru 0x20777c (0xdb7d bytes) 2024-09-18T01:03:37.132 INFO:teuthology.orchestra.run.smithi164.stdout:3092 read 0x6abc00 thru 0x6af19d (0x359e bytes) 2024-09-18T01:03:37.141 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:37.141 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:37.155 INFO:teuthology.orchestra.run.smithi164.stdout:3093 write 0xd41a400 thru 0xd41afc5 (0xbc6 bytes) 2024-09-18T01:03:37.162 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:37.162 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:37.169 INFO:teuthology.orchestra.run.smithi164.stdout:3095 trunc from 0xd41b000 to 0x1b2e600 2024-09-18T01:03:37.182 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:37.182 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:37.200 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:37.197+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:37.203 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:37.203 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:37.212 INFO:teuthology.orchestra.run.smithi164.stdout:3098 write 0x36db400 thru 0x36dfab7 (0x46b8 bytes) 2024-09-18T01:03:37.218 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:37.218 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:37.220 INFO:teuthology.orchestra.run.smithi164.stdout:3101 read 0x2b12800 thru 0x2b161d8 (0x39d9 bytes) 2024-09-18T01:03:37.220 INFO:teuthology.orchestra.run.smithi164.stdout:3102 read 0x388400 thru 0x3965c3 (0xe1c4 bytes) 2024-09-18T01:03:37.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:37.223 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:37.224 INFO:teuthology.orchestra.run.smithi164.stdout:3103 trunc from 0x36dfc00 to 0x2c3000 2024-09-18T01:03:37.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:37.242 INFO:teuthology.orchestra.run.smithi164.stdout:3104 read 0x125200 thru 0x128538 (0x3339 bytes) 2024-09-18T01:03:37.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:37.245 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:37.248 INFO:teuthology.orchestra.run.smithi164.stdout:3106 punch from 0x58200 to 0x62200, (0xa000 bytes) 2024-09-18T01:03:37.261 INFO:teuthology.orchestra.run.smithi164.stdout:3107 trunc from 0x2c3000 to 0xaf6c000 2024-09-18T01:03:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:3108 read 0x9f8d000 thru 0x9f9b08f (0xe090 bytes) 2024-09-18T01:03:37.280 INFO:teuthology.orchestra.run.smithi164.stdout:3111 trunc from 0xaf6c000 to 0x79d1a00 2024-09-18T01:03:37.290 INFO:teuthology.orchestra.run.smithi164.stdout:3112 punch from 0x3af000 to 0x3b5600, (0x6600 bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3113 punch from 0x2064600 to 0x2073e00, (0xf800 bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3114 read 0x5facc00 thru 0x5fb2d43 (0x6144 bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3115 read 0x356aa00 thru 0x356c593 (0x1b94 bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3116 read 0x3d99a00 thru 0x3da39d9 (0x9fda bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3117 write 0x1f0a800 thru 0x1f1706c (0xc86d bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3118 read 0x3bf0a00 thru 0x3bf41fa (0x37fb bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3120 write 0x6960400 thru 0x696a787 (0xa388 bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3121 read 0x1aab800 thru 0x1ab5b18 (0xa319 bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3124 read 0x44ac000 thru 0x44b1783 (0x5784 bytes) 2024-09-18T01:03:37.292 INFO:teuthology.orchestra.run.smithi164.stdout:3127 punch from 0x720200 to 0x72a200, (0xa000 bytes) 2024-09-18T01:03:37.294 INFO:teuthology.orchestra.run.smithi164.stdout:3128 trunc from 0x79d1a00 to 0x9524a00 2024-09-18T01:03:37.300 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:37.298+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:37.306 INFO:teuthology.orchestra.run.smithi164.stdout:3130 punch from 0x7f7f200 to 0x7f82000, (0x2e00 bytes) 2024-09-18T01:03:37.308 INFO:teuthology.orchestra.run.smithi164.stdout:3131 punch from 0x26f0c00 to 0x26fe800, (0xdc00 bytes) 2024-09-18T01:03:37.309 INFO:teuthology.orchestra.run.smithi164.stdout:3132 read 0x6e42600 thru 0x6e44d1b (0x271c bytes) 2024-09-18T01:03:37.309 INFO:teuthology.orchestra.run.smithi164.stdout:3133 write 0x4177c00 thru 0x417be30 (0x4231 bytes) 2024-09-18T01:03:37.309 INFO:teuthology.orchestra.run.smithi164.stdout:3135 read 0x84de800 thru 0x84e93e2 (0xabe3 bytes) 2024-09-18T01:03:37.309 INFO:teuthology.orchestra.run.smithi164.stdout:3136 read 0x77cf800 thru 0x77d0af7 (0x12f8 bytes) 2024-09-18T01:03:37.324 INFO:teuthology.orchestra.run.smithi164.stdout:3138 write 0xca16000 thru 0xca16477 (0x478 bytes) 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 64.961956752999996, 2024-09-18T01:03:37.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:37.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.342 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:37.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:37.345 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.963945447999997, 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:37.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.349 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:37.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:37.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:37.352 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 62.956268496, 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:37.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:37.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:37.359 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.858410546000002, 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:37.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:37.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:37.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 50.002329533999998, 2024-09-18T01:03:37.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:37.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:37.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 50.001960238000002, 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.373 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:37.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 50.000221392999997, 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.379 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:37.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:37.383 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 50.000149503000003, 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:37.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.388 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.912283492999997, 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:37.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:37.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.394 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.906092233999999, 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:37.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:37.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.904562548999998, 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.400 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:37.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:37.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.904182859999999, 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.406 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:37.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:37.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.412 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.888684456999997, 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.413 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:37.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.414 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:37.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:37.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:37.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:37.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.888477698000003, 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:37.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:37.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:37.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.888059941000002, 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.431 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.887850702999998, 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:37.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:37.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.436 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.887786769000002, 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:37.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.441 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.887038564000001, 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:37.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:37.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:37.446 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.628099759000001, 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:37.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:37.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 49.627236140999997, 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:37.453 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:37.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:37.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:37.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:37.459 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:37.460 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:37.458+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:37.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3140 write 0xe511000 thru 0xe518abe (0x7abf bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3141 read 0x929e200 thru 0x92a173f (0x3540 bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3144 trunc from 0xe518c00 to 0xc256a00 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3147 write 0x4b52800 thru 0x4b5cdbd (0xa5be bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3148 write 0x1291400 thru 0x1299696 (0x8297 bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3149 trunc from 0xc256a00 to 0x673e400 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3150 read 0x5c54400 thru 0x5c5a308 (0x5f09 bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3151 trunc from 0x673e400 to 0x3668400 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3154 read 0x118d000 thru 0x11947b8 (0x77b9 bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3157 write 0xbcfe00 thru 0xbdf37c (0xf57d bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3159 read 0x12f4600 thru 0x12fc373 (0x7d74 bytes) 2024-09-18T01:03:37.463 INFO:teuthology.orchestra.run.smithi164.stdout:3160 trunc from 0x3668400 to 0x1d3ba00 2024-09-18T01:03:37.464 INFO:teuthology.orchestra.run.smithi164.stdout:3161 flatten 2024-09-18T01:03:37.478 INFO:teuthology.orchestra.run.smithi164.stdout:3163 write 0xdbb9c00 thru 0xdbbcb67 (0x2f68 bytes) 2024-09-18T01:03:37.479 INFO:teuthology.orchestra.run.smithi164.stdout:3164 punch from 0x6003a00 to 0x6009e00, (0x6400 bytes) 2024-09-18T01:03:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:3166 read 0x79dcc00 thru 0x79eacc4 (0xe0c5 bytes) 2024-09-18T01:03:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:3167 write 0x4734e00 thru 0x473c56c (0x776d bytes) 2024-09-18T01:03:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:3168 read 0x2adfc00 thru 0x2ae931f (0x9720 bytes) 2024-09-18T01:03:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:3169 write 0xca9aa00 thru 0xcaa3db1 (0x93b2 bytes) 2024-09-18T01:03:37.480 INFO:teuthology.orchestra.run.smithi164.stdout:3170 read 0x63b5a00 thru 0x63b8f09 (0x350a bytes) 2024-09-18T01:03:37.493 INFO:teuthology.orchestra.run.smithi164.stdout:3171 trunc from 0xdbbcc00 to 0x2c5c000 2024-09-18T01:03:37.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:03:37.521 INFO:teuthology.orchestra.run.smithi164.stdout:3172 punch from 0x1892600 to 0x189d600, (0xb000 bytes) 2024-09-18T01:03:37.561 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:37.561+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:37.581 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:37.581 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:37.581 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:37.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:37.662 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:37.660+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:37.710 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:37.710 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:37.711 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:37.711 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:37.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:37.762 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:37.761+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:37.863 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:37.861+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:37.868 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2015 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4265984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.548245457, 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027955569999999999, 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2015 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318444+0000", 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318446+0000", 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.007e-06 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318453+0000", 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318458+0000", 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9910000000000002e-06 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346240+0000", 2024-09-18T01:03:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027782305 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346316+0000", 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5731999999999999e-05 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346399+0000", 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3137e-05 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.548224556999998, 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.548204048000002, 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.548184243000001, 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.548149602999999, 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.548128102, 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.548104924999997, 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2237 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4202496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.544136139999999, 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028056793, 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2237 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322553+0000", 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322555+0000", 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7740000000000001e-06 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322562+0000", 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.426e-06 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322567+0000", 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350490+0000", 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027923436999999999 2024-09-18T01:03:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350528+0000", 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7660999999999997e-05 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350610+0000", 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2150000000000005e-05 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.544116093, 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.544096539999998, 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.544076036, 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:37.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.544055172, 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:37.920 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:37.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.544035272000002, 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.544011183999999, 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:37.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.543979864000001, 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:37.934 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:37.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.543935257999998, 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:37.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.543904466999997, 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:37.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.543880981999997, 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:37.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.543851926999999, 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:37.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:37.957 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.543831167999997, 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:37.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:37.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:37.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:37.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:37.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:37.962 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:37.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:37.964 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:37.964+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:38.065 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:38.063+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:38.110 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:38.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:38.111 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:38.111 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:38.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:38.203 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:38.201+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:38.266 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:38.267 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:38.267 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:38.266+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:38.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:38.367 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:38.365+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:38.424 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.145211433999997, 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.140585530999999, 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.140507481999997, 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1292 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1774592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.083735756999999, 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050913359999999998, 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1292 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338599+0000", 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338601+0000", 2024-09-18T01:03:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.024e-06 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338610+0000", 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5469999999999997e-06 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338616+0000", 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9889999999999997e-06 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389393+0000", 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05077661 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389472+0000", 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9776000000000003e-05 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389512+0000", 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0000000000000003e-05 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.083711438999998, 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.083685574999997, 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.083663094999999, 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.083642058999999, 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.083614664999999, 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.083590360000002, 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.080190877, 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:38.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.080166317, 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:38.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.0801248, 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.080104298000002, 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:38.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.080083832, 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:38.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:38.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.080042491999997, 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:38.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:38.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.080022454000002, 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:38.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.079980122999999, 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:38.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.079959131000003, 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:38.514 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.079908160999999, 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:38.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.523 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:38.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:38.524 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:03:38.525 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:38.522+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:38.535 INFO:teuthology.orchestra.run.smithi164.stdout:3173 write 0x5c4b800 thru 0x5c58f1a (0xd71b bytes) 2024-09-18T01:03:38.537 INFO:teuthology.orchestra.run.smithi164.stdout:3176 write 0x4dee000 thru 0x4df36ac (0x56ad bytes) 2024-09-18T01:03:38.537 INFO:teuthology.orchestra.run.smithi164.stdout:3177 write 0x544f200 thru 0x545d577 (0xe378 bytes) 2024-09-18T01:03:38.573 INFO:teuthology.orchestra.run.smithi164.stdout:3179 write 0xc7c6000 thru 0xc7d0012 (0xa013 bytes) 2024-09-18T01:03:38.575 INFO:teuthology.orchestra.run.smithi164.stdout:3180 write 0x8c3ae00 thru 0x8c4a84b (0xfa4c bytes) 2024-09-18T01:03:38.575 INFO:teuthology.orchestra.run.smithi164.stdout:3182 write 0x2e12a00 thru 0x2e1833e (0x593f bytes) 2024-09-18T01:03:38.575 INFO:teuthology.orchestra.run.smithi164.stdout:3183 read 0xc6aa000 thru 0xc6b4b49 (0xab4a bytes) 2024-09-18T01:03:38.582 INFO:teuthology.orchestra.run.smithi164.stdout:3185 trunc from 0xc7d0200 to 0x8621200 2024-09-18T01:03:38.614 INFO:teuthology.orchestra.run.smithi164.stdout:3188 trunc from 0x8621200 to 0x50f7c00 2024-09-18T01:03:38.624 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:38.623+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:38.629 INFO:teuthology.orchestra.run.smithi164.stdout:3189 read 0x1161000 thru 0x117039f (0xf3a0 bytes) 2024-09-18T01:03:38.631 INFO:teuthology.orchestra.run.smithi164.stdout:3192 read 0x58a800 thru 0x58e1ed (0x39ee bytes) 2024-09-18T01:03:38.641 INFO:teuthology.orchestra.run.smithi164.stdout:3195 trunc from 0x50f7c00 to 0xbf6ba00 2024-09-18T01:03:38.655 INFO:teuthology.orchestra.run.smithi164.stdout:3196 read 0xb0e7a00 thru 0xb0f4763 (0xcd64 bytes) 2024-09-18T01:03:38.655 INFO:teuthology.orchestra.run.smithi164.stdout:3197 write 0x7f97000 thru 0x7f9cf48 (0x5f49 bytes) 2024-09-18T01:03:38.655 INFO:teuthology.orchestra.run.smithi164.stdout:3206 punch from 0x439aa00 to 0x439d000, (0x2600 bytes) 2024-09-18T01:03:38.670 INFO:teuthology.orchestra.run.smithi164.stdout:3207 write 0xe358600 thru 0xe363b76 (0xb577 bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3208 read 0xdca1a00 thru 0xdca5f6c (0x456d bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3209 write 0x1171c00 thru 0x117b7bc (0x9bbd bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3211 read 0xd593600 thru 0xd5a25bb (0xefbc bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3213 read 0xcf5ae00 thru 0xcf5b4c0 (0x6c1 bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3216 punch from 0x7c9e400 to 0x7ca8400, (0xa000 bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3217 write 0x5aea000 thru 0x5aeba98 (0x1a99 bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3218 write 0x82b0e00 thru 0x82c015b (0xf35c bytes) 2024-09-18T01:03:38.673 INFO:teuthology.orchestra.run.smithi164.stdout:3219 punch from 0x1162200 to 0x1164c00, (0x2a00 bytes) 2024-09-18T01:03:38.687 INFO:teuthology.orchestra.run.smithi164.stdout:3220 trunc from 0xe363c00 to 0x2852c00 2024-09-18T01:03:38.709 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5042.0:3335 3.18 3:1cf8d23b:::rbd_header.11034a3194e:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T01:03:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:38.708259+0000", 2024-09-18T01:03:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8872000000000001e-05, 2024-09-18T01:03:38.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6484999999999996e-05, 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5042", 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3335 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708259+0000", 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708259+0000", 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708260+0000", 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:38.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708263+0000", 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.382e-06 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708272+0000", 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.276e-06 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708276+0000", 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6319999999999999e-06 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708306+0000", 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0232000000000001e-05 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708342+0000", 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5564e-05 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5042.0:3336 3.18 3.dc4b1f38 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:38.708366+0000", 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9999518, 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8313999999999998e-05, 2024-09-18T01:03:38.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5042", 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3336 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708366+0000", 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708366+0000", 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708367+0000", 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9599999999999999e-07 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708370+0000", 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1489999999999998e-06 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708378+0000", 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5700000000000004e-06 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708381+0000", 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7680000000000002e-06 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:13588 3.18 3.dc4b1f38 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:38.708390+0000", 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.999928, 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7558000000000003e-05, 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:38.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13588 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708390+0000", 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708390+0000", 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708391+0000", 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0699999999999999e-06 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708396+0000", 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6079999999999998e-06 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708406+0000", 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0040999999999999e-05 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708413+0000", 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6490000000000002e-06 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:13589 3.18 3.dc4b1f38 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:38.708429+0000", 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9998894, 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119411, 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13589 2024-09-18T01:03:38.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708429+0000", 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:38.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708429+0000", 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708429+0000", 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708433+0000", 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6030000000000001e-06 2024-09-18T01:03:38.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708440+0000", 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:38.708446+0000", 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4689999999999996e-06 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:38.727 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 4 2024-09-18T01:03:38.727 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:38.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:38.729 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:38.726+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:38.730 INFO:teuthology.orchestra.run.smithi164.stdout:3221 write 0xaf06c00 thru 0xaf0e452 (0x7853 bytes) 2024-09-18T01:03:38.736 INFO:teuthology.orchestra.run.smithi164.stdout:3224 trunc from 0xaf0e600 to 0x6b3d600 2024-09-18T01:03:38.756 INFO:teuthology.orchestra.run.smithi164.stdout:3226 read 0x52a8a00 thru 0x52adbf4 (0x51f5 bytes) 2024-09-18T01:03:38.756 INFO:teuthology.orchestra.run.smithi164.stdout:3227 write 0x4268600 thru 0x42717e6 (0x91e7 bytes) 2024-09-18T01:03:38.778 INFO:teuthology.orchestra.run.smithi164.stdout:3229 write 0xa63c000 thru 0xa64a6d6 (0xe6d7 bytes) 2024-09-18T01:03:38.779 INFO:teuthology.orchestra.run.smithi164.stdout:3231 write 0x8169800 thru 0x817969b (0xfe9c bytes) 2024-09-18T01:03:38.779 INFO:teuthology.orchestra.run.smithi164.stdout:3232 punch from 0x747a400 to 0x747be00, (0x1a00 bytes) 2024-09-18T01:03:38.780 INFO:teuthology.orchestra.run.smithi164.stdout:3233 punch from 0x2e7a600 to 0x2e80e00, (0x6800 bytes) 2024-09-18T01:03:38.780 INFO:teuthology.orchestra.run.smithi164.stdout:3235 write 0x2f99c00 thru 0x2fa1fef (0x83f0 bytes) 2024-09-18T01:03:38.783 INFO:teuthology.orchestra.run.smithi164.stdout:3237 trunc from 0xa64a800 to 0x764f000 2024-09-18T01:03:38.822 INFO:teuthology.orchestra.run.smithi164.stdout:3238 trunc from 0x764f000 to 0x2f51800 2024-09-18T01:03:38.829 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:38.827+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:38.841 INFO:teuthology.orchestra.run.smithi164.stdout:3239 write 0x7ab200 thru 0x7b8cd8 (0xdad9 bytes) 2024-09-18T01:03:38.846 INFO:teuthology.orchestra.run.smithi164.stdout:3242 read 0x2435c00 thru 0x243de5f (0x8260 bytes) 2024-09-18T01:03:38.846 INFO:teuthology.orchestra.run.smithi164.stdout:3243 read 0x136f200 thru 0x137316c (0x3f6d bytes) 2024-09-18T01:03:38.851 INFO:teuthology.orchestra.run.smithi164.stdout:3244 trunc from 0x2f51800 to 0x7702a00 2024-09-18T01:03:38.865 INFO:teuthology.orchestra.run.smithi164.stdout:3247 punch from 0x14e5600 to 0x14e8200, (0x2c00 bytes) 2024-09-18T01:03:38.870 INFO:teuthology.orchestra.run.smithi164.stdout:3249 read 0x1509c00 thru 0x151874d (0xeb4e bytes) 2024-09-18T01:03:38.872 INFO:teuthology.orchestra.run.smithi164.stdout:3250 trunc from 0x7702a00 to 0x6a82200 2024-09-18T01:03:38.885 INFO:teuthology.orchestra.run.smithi164.stdout:3255 read 0x4a73e00 thru 0x4a75a30 (0x1c31 bytes) 2024-09-18T01:03:38.886 INFO:teuthology.orchestra.run.smithi164.stdout:3256 punch from 0x4befe00 to 0x4bfc800, (0xca00 bytes) 2024-09-18T01:03:38.886 INFO:teuthology.orchestra.run.smithi164.stdout:3257 read 0x28e3c00 thru 0x28e3cd8 (0xd9 bytes) 2024-09-18T01:03:38.887 INFO:teuthology.orchestra.run.smithi164.stdout:3259 clone 23 order 21 su 32768 sc 11 2024-09-18T01:03:38.896 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:38.896 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:38.897 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:38.897 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:38.897 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:38.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:38.950 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:38.948+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:39.014 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.277911222, 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:39.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:39.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.277891263999997, 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:39.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:39.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:39.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.277872322999997, 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:39.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:39.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:39.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.277850813999997, 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:39.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.277825086, 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.269093226999999, 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.269070513000003, 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.269051382999997, 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.269032537000001, 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.269013659000002, 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268991794000002, 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268968159000003, 2024-09-18T01:03:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268934508999997, 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268906653999998, 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268886831000003, 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268866396999996, 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268844317000003, 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268821995000003, 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268799598999998, 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.268779180000003, 2024-09-18T01:03:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:39.139 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:39.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:39.141 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:39.138+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:39.241 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:39.239+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:39.256 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:39.256 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:39.256 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:39.256 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:39.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:39.265 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:39.342 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:39.341+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:39.369 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:39.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:39.370 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:39.370 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:39.370 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:39.378 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:39.383 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:39.383 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:39.400 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:39.417 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:39.434 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:39.442 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:39.440+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:39.451 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:39.467 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:39.521 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:39.521 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:39.521 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T01:03:39.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-18T01:03:39.522 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:39.523 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:39.530 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:39.566 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:39.564+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:39.634 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:39.634 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:39.643 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:39.644 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:39.642+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:39.746 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:39.744+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:39.746 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:39.747 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:39.755 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:39.847 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:39.845+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:39.861 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:39.862 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:39.871 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:39.948 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:39.947+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:39.975 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:39.975 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:39.984 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:40.068 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:40.067+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:40.126 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:40.126 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:40.135 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:40.176 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:40.175+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:40.194 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-18T01:03:40.250 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:40.249+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:40.287 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:40.287 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:40.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:40.351 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:40.349+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:40.407 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:40.407 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:40.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:40.426 INFO:teuthology.orchestra.run.smithi164.stdout:3260 write 0x1b25000 thru 0x1b2a559 (0x555a bytes) 2024-09-18T01:03:40.449 INFO:teuthology.orchestra.run.smithi164.stdout:3263 write 0x86b2a00 thru 0x86bf285 (0xc886 bytes) 2024-09-18T01:03:40.450 INFO:teuthology.orchestra.run.smithi164.stdout:3266 read 0xfb2000 thru 0xfb63ea (0x43eb bytes) 2024-09-18T01:03:40.450 INFO:teuthology.orchestra.run.smithi164.stdout:3267 read 0x33ed800 thru 0x33f2faa (0x57ab bytes) 2024-09-18T01:03:40.451 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:40.450+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:40.472 INFO:teuthology.orchestra.run.smithi164.stdout:3270 write 0xe463200 thru 0xe46696e (0x376f bytes) 2024-09-18T01:03:40.472 INFO:teuthology.orchestra.run.smithi164.stdout:3275 punch from 0x8c2fe00 to 0x8c3f200, (0xf400 bytes) 2024-09-18T01:03:40.475 INFO:teuthology.orchestra.run.smithi164.stdout:3276 write 0x6ff7c00 thru 0x70031bf (0xb5c0 bytes) 2024-09-18T01:03:40.475 INFO:teuthology.orchestra.run.smithi164.stdout:3277 read 0x72b5c00 thru 0x72c5089 (0xf48a bytes) 2024-09-18T01:03:40.483 INFO:teuthology.orchestra.run.smithi164.stdout:3278 trunc from 0xe466a00 to 0x742f000 2024-09-18T01:03:40.507 INFO:teuthology.orchestra.run.smithi164.stdout:3279 write 0x22a8600 thru 0x22b0ba0 (0x85a1 bytes) 2024-09-18T01:03:40.508 INFO:teuthology.orchestra.run.smithi164.stdout:3281 read 0x4363600 thru 0x436d940 (0xa341 bytes) 2024-09-18T01:03:40.510 INFO:teuthology.orchestra.run.smithi164.stdout:3282 trunc from 0x742f000 to 0x5886000 2024-09-18T01:03:40.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:03:40.544 INFO:teuthology.orchestra.run.smithi031.stdout:moved 1 / 82 (1.21951%) 2024-09-18T01:03:40.544 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-18T01:03:40.544 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 2.93939 -> 3.13688 (expected baseline 3.62215) 2024-09-18T01:03:40.544 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.2 with 13 -> 13 pgs (0.792683 -> 0.792683 * mean) 2024-09-18T01:03:40.544 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.1 with 20 -> 20 pgs (1.21951 -> 1.21951 * mean) 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 0.05 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0005 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0006 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 weight 1.0000 -> 0.9500 2024-09-18T01:03:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 weight 1.0000 -> 0.9500 2024-09-18T01:03:40.552 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:40.551+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:40.553 INFO:teuthology.orchestra.run.smithi164.stdout:3284 punch from 0x4826200 to 0x4826a00, (0x800 bytes) 2024-09-18T01:03:40.556 INFO:teuthology.orchestra.run.smithi164.stdout:3285 punch from 0x4531c00 to 0x4536200, (0x4600 bytes) 2024-09-18T01:03:40.556 INFO:teuthology.orchestra.run.smithi164.stdout:3288 punch from 0x23c6800 to 0x23ca400, (0x3c00 bytes) 2024-09-18T01:03:40.562 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T01:03:40.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-18T01:03:40.586 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:40.587 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:40.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:40.653 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:40.652+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:40.747 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:40.747 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:40.753 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:40.752+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:40.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:40.855 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:40.854+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:40.904 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:40.904 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:40.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:40.957 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:40.955+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:41.020 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:41.020 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:41.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:41.057 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:41.056+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:41.158 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:41.157+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:41.175 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:41.175 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:41.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:41.259 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:41.257+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:41.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:41.361 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:41.360+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:41.447 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:41.447 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:41.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:41.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:41.499 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:41.497+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:41.551 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:moved 6 / 82 (7.31707%) 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 3.13688 -> 4.07922 (expected baseline 3.62215) 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.2 with 13 -> 11 pgs (0.792683 -> 0.670732 * mean) 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.1 with 20 -> 21 pgs (1.21951 -> 1.28049 * mean) 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 1 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0005 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0006 2024-09-18T01:03:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 weight 0.9500 -> 0.6882 2024-09-18T01:03:41.555 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 weight 0.9500 -> 0.8100 2024-09-18T01:03:41.563 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:41.561+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:41.572 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:41.572 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:41.575 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:03:41.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-18T01:03:41.582 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:41.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:41.628 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:41.647 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:41.648 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:41.664 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:41.664 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:41.665 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:41.663+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:41.681 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:41.681 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:41.695 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:41.696 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:41.696 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:41.696 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:41.698 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:41.698 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:41.705 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:41.768 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:41.766+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:41.848 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:41.848 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:41.848 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:41.848 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:41.848 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:41.856 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:41.900 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:41.899+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:41.913 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:03:41.912+0000 7f0a110b7640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-18T01:03:41.914 INFO:teuthology.orchestra.run.smithi031.stdout:moved 9 / 82 (10.9756%) 2024-09-18T01:03:41.914 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-18T01:03:41.914 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 4.07922 -> 6.97424 (expected baseline 3.62215) 2024-09-18T01:03:41.914 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.2 with 11 -> 7 pgs (0.670732 -> 0.426829 * mean) 2024-09-18T01:03:41.914 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.1 with 21 -> 24 pgs (1.28049 -> 1.46341 * mean) 2024-09-18T01:03:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 3 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0005 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0006 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 weight 0.6882 -> 0.4985 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 weight 0.8100 -> 0.6906 2024-09-18T01:03:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-18T01:03:41.927 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:03:41.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-18T01:03:41.960 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:41.960 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.914988575999999, 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.961 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:03:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.914977425000004, 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.966 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:41.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:03:41.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.914965594999998, 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:03:41.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:41.975 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.914948912, 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:41.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:03:41.979 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.914937674999997, 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.980 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:03:41.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.984 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.913615569999997, 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:41.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:03:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.913602855000001, 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:03:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.913557871000002, 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:03:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.913530246999997, 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.002 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.909982868, 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:03:42.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.909622761000001, 2024-09-18T01:03:42.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:03:42.011 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.909596647999997, 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.012 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:03:42.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:03:42.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.907417260000003, 2024-09-18T01:03:42.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:03:42.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.020 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.907407464999999, 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:03:42.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.907396685999998, 2024-09-18T01:03:42.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:42.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.029 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.907376890000002, 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.030 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:03:42.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:42.034 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.905593369999998, 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:03:42.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.905566532000002, 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.039 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:03:42.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.043 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 39.905555984000003, 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:42.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:03:42.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:42.048 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 4.4336291079999999, 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:42.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:03:42.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.054 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.054 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:42.056 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:42.056 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:42.054+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:42.156 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:42.154+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:42.200 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:42.201 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:42.201 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:42.201 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:42.209 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:42.257 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:42.256+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:42.312 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:42.312 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:42.312 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:42.312 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:42.312 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:42.320 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:42.340 INFO:tasks.ceph.mgr.x.smithi098.stderr:2024-09-18T01:03:42.339+0000 7f0a110b7640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-18T01:03:42.341 INFO:teuthology.orchestra.run.smithi031.stdout:moved 9 / 82 (10.9756%) 2024-09-18T01:03:42.341 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-18T01:03:42.341 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 4.07922 -> 6.97424 (expected baseline 3.62215) 2024-09-18T01:03:42.341 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.2 with 11 -> 7 pgs (0.670732 -> 0.426829 * mean) 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.1 with 21 -> 24 pgs (1.28049 -> 1.46341 * mean) 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 1 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0005 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0006 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 weight 0.6882 -> 0.4985 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 weight 0.8100 -> 0.6906 2024-09-18T01:03:42.342 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-18T01:03:42.353 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T01:03:42.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-18T01:03:42.395 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:42.393+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:42.445 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.445 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.147173125000002, 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.446 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:03:42.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.147149008, 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.450 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:03:42.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.147069768000001, 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:03:42.455 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:42.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.458 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.459 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.147033549, 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.460 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:03:42.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.147004936000002, 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.465 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:42.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:03:42.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.468 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:03:42.469 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.146984070999999, 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.470 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:42.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:42.475 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.146366354000001, 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:42.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:03:42.480 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.146337086999999, 2024-09-18T01:03:42.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.486 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.146270796000003, 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.487 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:03:42.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.491 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.146150028999998, 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:03:42.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.496 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145931582999999, 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.497 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:03:42.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:03:42.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.501 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145910221999998, 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:42.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:03:42.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145756282999997, 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:03:42.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145736565, 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.512 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:03:42.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145716255000004, 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.517 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:42.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145696368999999, 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.522 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:42.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:03:42.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:03:42.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.527 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.14567641, 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:03:42.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:03:42.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.532 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145656412000001, 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.533 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:03:42.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.537 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145636744999997, 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:03:42.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.145616842999999, 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.543 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:03:42.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:03:42.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.549 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:42.551 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:42.548+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:42.551 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:42.651 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:42.650+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:42.667 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:42.667 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:42.676 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:42.692 INFO:teuthology.orchestra.run.smithi031.stdout:moved 0 / 82 (0%) 2024-09-18T01:03:42.692 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 4.07922 -> 4.07922 (expected baseline 3.62215) 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.2 with 11 -> 11 pgs (0.670732 -> 0.670732 * mean) 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.1 with 21 -> 21 pgs (1.28049 -> 1.28049 * mean) 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:oload 1000 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 3 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0005 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0052 2024-09-18T01:03:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-18T01:03:42.760 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:42.759+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:42.818 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:42.819 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:42.827 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:42.854 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:42.853+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:42.955 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:42.954+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.596009906000006, 2024-09-18T01:03:42.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:42.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:42.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 69.597998601, 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:42.979 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:42.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:42.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.590321649000003, 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:42.986 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:42.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:42.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 59.492463698999998, 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:42.993 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:42.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:42.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:42.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:42.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.000 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.636382687000001, 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:43.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:43.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.636013390999999, 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.007 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.012 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.634274546, 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.013 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.015 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.017 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.634202655999999, 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:43.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:43.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.546336646, 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.022 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:43.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:43.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:43.028 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.540145387000003, 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:43.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:43.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:43.033 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.538615702000001, 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:43.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:43.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.039 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.538236013000002, 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.040 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:43.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:43.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:43.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.52273761, 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.046 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:43.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.522530850999999, 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:43.052 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:43.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:43.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:43.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.522113093999998, 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:43.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:43.063 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.521903856000002, 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:43.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.521839921999998, 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:43.068 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:43.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.072 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.521091716999997, 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:43.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:43.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.077 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.262152911999998, 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.078 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:43.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:43.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 55.261289294000001, 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:43.084 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:43.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:43.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:43.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:43.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.089 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.090 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:43.090 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:43.090 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:43.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:43.092 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:43.090+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:43.192 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:43.190+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3.b6903b57 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.62548195600000001, 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.62550210299999998, 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:03:43.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:03:43.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3.b6903b57 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.62556566499999999, 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.62563110799999999, 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-18T01:03:43.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:03:43.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 2 2024-09-18T01:03:43.215 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:43.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:43.300 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:43.298+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:43.325 INFO:teuthology.orchestra.run.smithi164.stdout:3290 write 0xc400600 thru 0xc4010b9 (0xaba bytes) 2024-09-18T01:03:43.326 INFO:teuthology.orchestra.run.smithi164.stdout:3293 read 0x6340800 thru 0x634e35a (0xdb5b bytes) 2024-09-18T01:03:43.346 INFO:teuthology.orchestra.run.smithi164.stdout:3295 trunc from 0xc401200 to 0x1cb1800 2024-09-18T01:03:43.394 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:43.392+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:43.395 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:43.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:43.495 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:43.493+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:43.562 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.242276249, 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:43.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:43.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.242255739999997, 2024-09-18T01:03:43.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:43.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.242235934999997, 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:43.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.578 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.242201295000001, 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.579 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:43.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:43.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.242179794000002, 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:43.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.242156616999999, 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:43.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.238167785000002, 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:43.597 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.238148232, 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:43.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.238127728000002, 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.238106864000002, 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:43.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.238086963999997, 2024-09-18T01:03:43.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:43.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:43.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.238062876000001, 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.238031556000003, 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:43.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:43.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:43.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.23798695, 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:43.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:43.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:43.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.237956158999999, 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:43.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.237932674, 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:43.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.237903619000001, 2024-09-18T01:03:43.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:43.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:43.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.237882859999999, 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:43.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.979942113, 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:03:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:03:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.97985840400000002, 2024-09-18T01:03:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:03:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:43.660 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:43.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:03:43.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:43.663 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:43.659+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:43.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:43.763 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:43.762+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:43.812 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:43.813 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:43.813 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:43.813 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:43.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:43.865 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:43.863+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:43.876 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:43.876 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:43.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:43.910 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:43.927 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:43.927 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:43.933 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:43.933 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:43.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:43.945 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:43.945 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:43.962 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:43.962 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:43.964 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:43.963+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:43.979 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:43.979 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:44.053 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.054 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:44.065 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:44.063+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:44.166 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:44.165+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:44.170 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.170 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:44.266 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:44.264+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:44.286 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:03:44.290 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.290 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:44.386 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:44.384+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:44.450 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.450 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.459 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:44.503 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:44.502+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:44.568 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:44.567+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:44.573 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.573 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.582 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:44.669 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:44.668+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:44.725 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.726 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.734 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:44.778 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:44.777+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:44.837 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.837 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.846 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:44.870 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:44.869+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:44.971 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:44.970+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:44.989 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:44.989 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:44.999 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:45.072 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:45.071+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:45.102 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:45.102 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:45.110 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:45.173 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:45.172+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:45.212 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:45.212 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:45.220 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:45.274 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:45.273+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:45.326 INFO:teuthology.orchestra.run.smithi164.stdout:3296 punch from 0x1571200 to 0x1571800, (0x600 bytes) 2024-09-18T01:03:45.357 INFO:teuthology.orchestra.run.smithi164.stdout:3298 write 0xce52a00 thru 0xce60d6c (0xe36d bytes) 2024-09-18T01:03:45.358 INFO:teuthology.orchestra.run.smithi164.stdout:3300 read 0x6f1fe00 thru 0x6f25768 (0x5969 bytes) 2024-09-18T01:03:45.358 INFO:teuthology.orchestra.run.smithi164.stdout:3301 punch from 0xd89c00 to 0xd8d800, (0x3c00 bytes) 2024-09-18T01:03:45.360 INFO:teuthology.orchestra.run.smithi164.stdout:3302 write 0x30d2400 thru 0x30d848d (0x608e bytes) 2024-09-18T01:03:45.360 INFO:teuthology.orchestra.run.smithi164.stdout:3304 write 0x616e600 thru 0x6172447 (0x3e48 bytes) 2024-09-18T01:03:45.360 INFO:teuthology.orchestra.run.smithi164.stdout:3305 read 0x8f57200 thru 0x8f61538 (0xa339 bytes) 2024-09-18T01:03:45.360 INFO:teuthology.orchestra.run.smithi164.stdout:3307 read 0xab2d000 thru 0xab3bded (0xedee bytes) 2024-09-18T01:03:45.361 INFO:teuthology.orchestra.run.smithi164.stdout:3312 write 0x7d45a00 thru 0x7d5558b (0xfb8c bytes) 2024-09-18T01:03:45.361 INFO:teuthology.orchestra.run.smithi164.stdout:3313 punch from 0xc60c600 to 0xc61b800, (0xf200 bytes) 2024-09-18T01:03:45.362 INFO:teuthology.orchestra.run.smithi164.stdout:3314 read 0x408b600 thru 0x4094e33 (0x9834 bytes) 2024-09-18T01:03:45.362 INFO:teuthology.orchestra.run.smithi164.stdout:3315 write 0x7973c00 thru 0x797bddc (0x81dd bytes) 2024-09-18T01:03:45.362 INFO:teuthology.orchestra.run.smithi164.stdout:3316 read 0x3babe00 thru 0x3bb3358 (0x7559 bytes) 2024-09-18T01:03:45.362 INFO:teuthology.orchestra.run.smithi164.stdout:3319 write 0x69d3800 thru 0x69dca8e (0x928f bytes) 2024-09-18T01:03:45.362 INFO:teuthology.orchestra.run.smithi164.stdout:3320 punch from 0xa16a200 to 0xa176400, (0xc200 bytes) 2024-09-18T01:03:45.362 INFO:teuthology.orchestra.run.smithi164.stdout:3321 write 0xc5b8200 thru 0xc5c0259 (0x805a bytes) 2024-09-18T01:03:45.374 INFO:teuthology.orchestra.run.smithi164.stdout:3323 trunc from 0xce60e00 to 0x481da00 2024-09-18T01:03:45.374 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:45.373+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:45.378 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:45.378 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:45.388 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:45.407 INFO:teuthology.orchestra.run.smithi164.stdout:3326 punch from 0x1463800 to 0x1471e00, (0xe600 bytes) 2024-09-18T01:03:45.429 INFO:teuthology.orchestra.run.smithi164.stdout:3328 write 0x7daae00 thru 0x7dadc3a (0x2e3b bytes) 2024-09-18T01:03:45.429 INFO:teuthology.orchestra.run.smithi164.stdout:3330 write 0xe47200 thru 0xe56790 (0xf591 bytes) 2024-09-18T01:03:45.429 INFO:teuthology.orchestra.run.smithi164.stdout:3331 punch from 0x3da5600 to 0x3db4c00, (0xf600 bytes) 2024-09-18T01:03:45.434 INFO:teuthology.orchestra.run.smithi164.stdout:3332 trunc from 0x7dade00 to 0xbf21200 2024-09-18T01:03:45.459 INFO:teuthology.orchestra.run.smithi164.stdout:3334 trunc from 0xbf21200 to 0x7d03600 2024-09-18T01:03:45.476 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:45.475+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:45.481 INFO:teuthology.orchestra.run.smithi164.stdout:3335 trunc from 0x7d03600 to 0xcc2ce00 2024-09-18T01:03:45.495 INFO:teuthology.orchestra.run.smithi164.stdout:3337 punch from 0x6b9de00 to 0x6b9e800, (0xa00 bytes) 2024-09-18T01:03:45.508 INFO:teuthology.orchestra.run.smithi164.stdout:3338 trunc from 0xcc2ce00 to 0x1f3dc00 2024-09-18T01:03:45.530 INFO:teuthology.orchestra.run.smithi164.stdout:3339 punch from 0x1427800 to 0x1429600, (0x1e00 bytes) 2024-09-18T01:03:45.543 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:45.543 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:45.547 INFO:teuthology.orchestra.run.smithi164.stdout:3342 write 0x4ae4200 thru 0x4aee348 (0xa149 bytes) 2024-09-18T01:03:45.547 INFO:teuthology.orchestra.run.smithi164.stdout:3343 write 0x1b9ea00 thru 0x1ba6a2b (0x802c bytes) 2024-09-18T01:03:45.547 INFO:teuthology.orchestra.run.smithi164.stdout:3344 write 0x3d54400 thru 0x3d5bd8d (0x798e bytes) 2024-09-18T01:03:45.547 INFO:teuthology.orchestra.run.smithi164.stdout:3345 punch from 0x2d9ea00 to 0x2da8000, (0x9600 bytes) 2024-09-18T01:03:45.548 INFO:teuthology.orchestra.run.smithi164.stdout:3346 punch from 0x3ad5a00 to 0x3adf800, (0x9e00 bytes) 2024-09-18T01:03:45.548 INFO:teuthology.orchestra.run.smithi164.stdout:3347 write 0x4699200 thru 0x469c6fe (0x34ff bytes) 2024-09-18T01:03:45.548 INFO:teuthology.orchestra.run.smithi164.stdout:3349 read 0xd83600 thru 0xd87d48 (0x4749 bytes) 2024-09-18T01:03:45.548 INFO:teuthology.orchestra.run.smithi164.stdout:3350 read 0x10a9a00 thru 0x10b2f6d (0x956e bytes) 2024-09-18T01:03:45.549 INFO:teuthology.orchestra.run.smithi164.stdout:3354 punch from 0x2b34000 to 0x2b35800, (0x1800 bytes) 2024-09-18T01:03:45.549 INFO:teuthology.orchestra.run.smithi164.stdout:3355 read 0xafcc00 thru 0xb05634 (0x8a35 bytes) 2024-09-18T01:03:45.551 INFO:teuthology.orchestra.run.smithi164.stdout:3357 trunc from 0x4aee400 to 0x5fbce00 2024-09-18T01:03:45.553 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:45.569 INFO:teuthology.orchestra.run.smithi164.stdout:3359 read 0xe9d800 thru 0xea9cb6 (0xc4b7 bytes) 2024-09-18T01:03:45.579 INFO:teuthology.orchestra.run.smithi164.stdout:3360 write 0x6a03e00 thru 0x6a13245 (0xf446 bytes) 2024-09-18T01:03:45.580 INFO:teuthology.orchestra.run.smithi164.stdout:3361 write 0x32c2a00 thru 0x32cf980 (0xcf81 bytes) 2024-09-18T01:03:45.598 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:45.597+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:45.604 INFO:teuthology.orchestra.run.smithi164.stdout:3364 write 0xe68c600 thru 0xe693e68 (0x7869 bytes) 2024-09-18T01:03:45.605 INFO:teuthology.orchestra.run.smithi164.stdout:3365 trunc from 0xe694000 to 0xe894400 2024-09-18T01:03:45.619 INFO:teuthology.orchestra.run.smithi164.stdout:3366 read 0x366f400 thru 0x367e9ee (0xf5ef bytes) 2024-09-18T01:03:45.622 INFO:teuthology.orchestra.run.smithi164.stdout:3367 write 0x773c800 thru 0x774c318 (0xfb19 bytes) 2024-09-18T01:03:45.622 INFO:teuthology.orchestra.run.smithi164.stdout:3368 write 0xc5bb800 thru 0xc5cb613 (0xfe14 bytes) 2024-09-18T01:03:45.622 INFO:teuthology.orchestra.run.smithi164.stdout:3369 write 0x1538400 thru 0x153f6d6 (0x72d7 bytes) 2024-09-18T01:03:45.622 INFO:teuthology.orchestra.run.smithi164.stdout:3370 write 0xa4c3c00 thru 0xa4cc5d2 (0x89d3 bytes) 2024-09-18T01:03:45.622 INFO:teuthology.orchestra.run.smithi164.stdout:3371 read 0x4904000 thru 0x4910d83 (0xcd84 bytes) 2024-09-18T01:03:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:3373 write 0x9678800 thru 0x9688627 (0xfe28 bytes) 2024-09-18T01:03:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:3374 read 0xe4eca00 thru 0xe4f6904 (0x9f05 bytes) 2024-09-18T01:03:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:3375 read 0xbd2000 thru 0xbda97d (0x897e bytes) 2024-09-18T01:03:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:3376 write 0xac83e00 thru 0xac92cd5 (0xeed6 bytes) 2024-09-18T01:03:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:3381 write 0xbb7a800 thru 0xbb86dda (0xc5db bytes) 2024-09-18T01:03:45.623 INFO:teuthology.orchestra.run.smithi164.stdout:3382 write 0x2b9fe00 thru 0x2bad634 (0xd835 bytes) 2024-09-18T01:03:45.624 INFO:teuthology.orchestra.run.smithi164.stdout:3384 write 0x851dc00 thru 0x8526508 (0x8909 bytes) 2024-09-18T01:03:45.624 INFO:teuthology.orchestra.run.smithi164.stdout:3386 punch from 0x40ed200 to 0x40f2000, (0x4e00 bytes) 2024-09-18T01:03:45.625 INFO:teuthology.orchestra.run.smithi164.stdout:3388 read 0x514e800 thru 0x515d871 (0xf072 bytes) 2024-09-18T01:03:45.639 INFO:teuthology.orchestra.run.smithi164.stdout:3390 trunc from 0xe894400 to 0x299ae00 2024-09-18T01:03:45.674 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:45.674 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:45.676 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:45.675+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:45.681 INFO:teuthology.orchestra.run.smithi164.stdout:3392 read 0x1bbea00 thru 0x1bc4ee0 (0x64e1 bytes) 2024-09-18T01:03:45.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:45.694 INFO:teuthology.orchestra.run.smithi164.stdout:3393 trunc from 0x299ae00 to 0x84fa800 2024-09-18T01:03:45.702 INFO:teuthology.orchestra.run.smithi164.stdout:3396 read 0x4bd1e00 thru 0x4bd57df (0x39e0 bytes) 2024-09-18T01:03:45.714 INFO:teuthology.orchestra.run.smithi164.stdout:3398 write 0x9eea600 thru 0x9ef9570 (0xef71 bytes) 2024-09-18T01:03:45.715 INFO:teuthology.orchestra.run.smithi164.stdout:3399 write 0xec7000 thru 0xed6531 (0xf532 bytes) 2024-09-18T01:03:45.718 INFO:teuthology.orchestra.run.smithi164.stdout:3401 trunc from 0x9ef9600 to 0x71ed200 2024-09-18T01:03:45.748 INFO:teuthology.orchestra.run.smithi164.stdout:3403 trunc from 0x71ed200 to 0x2068a00 2024-09-18T01:03:45.760 INFO:teuthology.orchestra.run.smithi164.stdout:3405 punch from 0x2de00 to 0x3c600, (0xe800 bytes) 2024-09-18T01:03:45.762 INFO:teuthology.orchestra.run.smithi164.stdout:3406 read 0x9d2000 thru 0x9d3f54 (0x1f55 bytes) 2024-09-18T01:03:45.765 INFO:teuthology.orchestra.run.smithi164.stdout:3408 punch from 0x1c58e00 to 0x1c62e00, (0xa000 bytes) 2024-09-18T01:03:45.775 INFO:teuthology.orchestra.run.smithi164.stdout:3409 write 0x2d87800 thru 0x2d93fdd (0xc7de bytes) 2024-09-18T01:03:45.777 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:45.776+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:45.792 INFO:teuthology.orchestra.run.smithi164.stdout:3411 write 0x4caa600 thru 0x4cb68a9 (0xc2aa bytes) 2024-09-18T01:03:45.805 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:45.805 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:45.814 INFO:teuthology.orchestra.run.smithi164.stdout:3414 write 0xb2e7c00 thru 0xb2f5638 (0xda39 bytes) 2024-09-18T01:03:45.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:45.815 INFO:teuthology.orchestra.run.smithi164.stdout:3417 read 0x404800 thru 0x4110d7 (0xc8d8 bytes) 2024-09-18T01:03:45.816 INFO:teuthology.orchestra.run.smithi164.stdout:3419 punch from 0x3687e00 to 0x3692600, (0xa800 bytes) 2024-09-18T01:03:45.817 INFO:teuthology.orchestra.run.smithi164.stdout:3420 read 0x5876e00 thru 0x588019e (0x939f bytes) 2024-09-18T01:03:45.817 INFO:teuthology.orchestra.run.smithi164.stdout:3421 write 0x11ccc00 thru 0x11dc7c0 (0xfbc1 bytes) 2024-09-18T01:03:45.827 INFO:teuthology.orchestra.run.smithi164.stdout:3426 trunc from 0xb2f5800 to 0x46e2e00 2024-09-18T01:03:45.852 INFO:teuthology.orchestra.run.smithi164.stdout:3431 read 0x14b3600 thru 0x14bddb5 (0xa7b6 bytes) 2024-09-18T01:03:45.852 INFO:teuthology.orchestra.run.smithi164.stdout:3432 read 0xc45a00 thru 0xc4f1af (0x97b0 bytes) 2024-09-18T01:03:45.854 INFO:teuthology.orchestra.run.smithi164.stdout:3433 read 0x2297800 thru 0x22a3f43 (0xc744 bytes) 2024-09-18T01:03:45.854 INFO:teuthology.orchestra.run.smithi164.stdout:3434 read 0x2622a00 thru 0x263136a (0xe96b bytes) 2024-09-18T01:03:45.878 INFO:teuthology.orchestra.run.smithi164.stdout:3435 write 0xdab6800 thru 0xdab86d5 (0x1ed6 bytes) 2024-09-18T01:03:45.878 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:45.877+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:45.892 INFO:teuthology.orchestra.run.smithi164.stdout:3438 trunc from 0xdab8800 to 0x1ce4e00 2024-09-18T01:03:45.932 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:45.932 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:45.940 INFO:teuthology.orchestra.run.smithi164.stdout:3439 write 0xdf2d200 thru 0xdf33af8 (0x68f9 bytes) 2024-09-18T01:03:45.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:45.956 INFO:teuthology.orchestra.run.smithi164.stdout:3440 trunc from 0xdf33c00 to 0x12b5c00 2024-09-18T01:03:45.979 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:45.978+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:45.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:46.051 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:46.051 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:46.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:46.104 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:46.103+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:46.115 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:46.133 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:46.133 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:46.151 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:46.151 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:46.173 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:46.176 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:46.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:46.176 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:46.176 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:46.180 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:46.179+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:46.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:46.189 INFO:teuthology.orchestra.run.smithi164.stdout:3441 write 0x5bae400 thru 0x5bb93b7 (0xafb8 bytes) 2024-09-18T01:03:46.192 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:46.192 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:46.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:46.213 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:46.221 INFO:teuthology.orchestra.run.smithi164.stdout:3442 write 0xb239000 thru 0xb2404f6 (0x74f7 bytes) 2024-09-18T01:03:46.221 INFO:teuthology.orchestra.run.smithi164.stdout:3444 read 0xf7ee00 thru 0xf8a458 (0xb659 bytes) 2024-09-18T01:03:46.224 INFO:teuthology.orchestra.run.smithi164.stdout:3445 write 0x52c8a00 thru 0x52d3a58 (0xb059 bytes) 2024-09-18T01:03:46.224 INFO:teuthology.orchestra.run.smithi164.stdout:3446 write 0xa4c8400 thru 0xa4c9f7f (0x1b80 bytes) 2024-09-18T01:03:46.244 INFO:teuthology.orchestra.run.smithi164.stdout:3447 write 0xe8e4e00 thru 0xe8f2ad7 (0xdcd8 bytes) 2024-09-18T01:03:46.251 INFO:teuthology.orchestra.run.smithi164.stdout:3448 read 0x544a000 thru 0x5457f5e (0xdf5f bytes) 2024-09-18T01:03:46.251 INFO:teuthology.orchestra.run.smithi164.stdout:3450 punch from 0xf0fe00 to 0xf18e00, (0x9000 bytes) 2024-09-18T01:03:46.253 INFO:teuthology.orchestra.run.smithi164.stdout:3451 write 0x9110c00 thru 0x9111e51 (0x1252 bytes) 2024-09-18T01:03:46.254 INFO:teuthology.orchestra.run.smithi164.stdout:3452 read 0x90b7800 thru 0x90bab6e (0x336f bytes) 2024-09-18T01:03:46.254 INFO:teuthology.orchestra.run.smithi164.stdout:3453 write 0x4956000 thru 0x49589b6 (0x29b7 bytes) 2024-09-18T01:03:46.254 INFO:teuthology.orchestra.run.smithi164.stdout:3454 read 0x41e3e00 thru 0x41e5fac (0x21ad bytes) 2024-09-18T01:03:46.268 INFO:teuthology.orchestra.run.smithi164.stdout:3455 trunc from 0xe8f2c00 to 0x1c7d400 2024-09-18T01:03:46.281 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:46.280+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:46.294 INFO:teuthology.orchestra.run.smithi164.stdout:3456 punch from 0x1910000 to 0x1915400, (0x5400 bytes) 2024-09-18T01:03:46.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:03:46.314 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:46.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:46.315 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:46.315 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:46.315 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:46.315 INFO:teuthology.orchestra.run.smithi164.stdout:3457 write 0x34a7e00 thru 0x34aca80 (0x4c81 bytes) 2024-09-18T01:03:46.315 INFO:teuthology.orchestra.run.smithi164.stdout:3458 punch from 0x1109e00 to 0x110de00, (0x4000 bytes) 2024-09-18T01:03:46.317 INFO:teuthology.orchestra.run.smithi164.stdout:3459 write 0xb9aa00 thru 0xba6337 (0xb938 bytes) 2024-09-18T01:03:46.318 INFO:teuthology.orchestra.run.smithi164.stdout:3463 read 0x3139800 thru 0x313ceab (0x36ac bytes) 2024-09-18T01:03:46.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout:3467 write 0xe595000 thru 0xe5a0600 (0xb601 bytes) 2024-09-18T01:03:46.342 INFO:teuthology.orchestra.run.smithi164.stdout:3468 write 0xdba0e00 thru 0xdbadbe9 (0xcdea bytes) 2024-09-18T01:03:46.347 INFO:teuthology.orchestra.run.smithi164.stdout:3469 trunc from 0xe5a0800 to 0x9e7e400 2024-09-18T01:03:46.370 INFO:teuthology.orchestra.run.smithi164.stdout:3470 punch from 0x5213600 to 0x5216e00, (0x3800 bytes) 2024-09-18T01:03:46.391 INFO:teuthology.orchestra.run.smithi164.stdout:3472 write 0xa783c00 thru 0xa786503 (0x2904 bytes) 2024-09-18T01:03:46.393 INFO:teuthology.orchestra.run.smithi164.stdout:3473 read 0x793c00 thru 0x79dff4 (0xa3f5 bytes) 2024-09-18T01:03:46.394 INFO:teuthology.orchestra.run.smithi164.stdout:3474 read 0x9951e00 thru 0x9955361 (0x3562 bytes) 2024-09-18T01:03:46.397 INFO:teuthology.orchestra.run.smithi164.stdout:3475 trunc from 0xa786600 to 0x821fa00 2024-09-18T01:03:46.410 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:46.409+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:46.418 INFO:teuthology.orchestra.run.smithi164.stdout:3476 trunc from 0x821fa00 to 0x53d1a00 2024-09-18T01:03:46.445 INFO:teuthology.orchestra.run.smithi164.stdout:3480 trunc from 0x53d1a00 to 0xd911000 2024-09-18T01:03:46.453 INFO:teuthology.orchestra.run.smithi164.stdout:3483 punch from 0x105a800 to 0x1067600, (0xce00 bytes) 2024-09-18T01:03:46.456 INFO:teuthology.orchestra.run.smithi164.stdout:3484 write 0x966d800 thru 0x96751e5 (0x79e6 bytes) 2024-09-18T01:03:46.457 INFO:teuthology.orchestra.run.smithi164.stdout:3493 read 0x3f9c200 thru 0x3fa3cf3 (0x7af4 bytes) 2024-09-18T01:03:46.457 INFO:teuthology.orchestra.run.smithi164.stdout:3496 read 0x9737400 thru 0x9739a61 (0x2662 bytes) 2024-09-18T01:03:46.457 INFO:teuthology.orchestra.run.smithi164.stdout:3497 write 0x799c400 thru 0x79a88a6 (0xc4a7 bytes) 2024-09-18T01:03:46.457 INFO:teuthology.orchestra.run.smithi164.stdout:3499 read 0x2fb3400 thru 0x2fb430b (0xf0c bytes) 2024-09-18T01:03:46.457 INFO:teuthology.orchestra.run.smithi164.stdout:3502 read 0x6af800 thru 0x6be489 (0xec8a bytes) 2024-09-18T01:03:46.464 INFO:teuthology.orchestra.run.smithi164.stdout:3506 trunc from 0xd911000 to 0x85f5c00 2024-09-18T01:03:46.483 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:46.482+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:46.485 INFO:teuthology.orchestra.run.smithi164.stdout:3508 punch from 0x34c3e00 to 0x34cd600, (0x9800 bytes) 2024-09-18T01:03:46.487 INFO:teuthology.orchestra.run.smithi164.stdout:3509 read 0x1b9800 thru 0x1c3288 (0x9a89 bytes) 2024-09-18T01:03:46.493 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:46.493 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:46.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:46.493 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:46.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.213005209000002, 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:46.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:46.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:46.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:46.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:46.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:46.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.208379305999998, 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:46.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:46.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.208301257000002, 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:46.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:46.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:46.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:46.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.151505213999997, 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:46.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.151479350000002, 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:46.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:46.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:46.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:46.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.151456869999997, 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:46.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.151435833999997, 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:46.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:46.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.151408439999997, 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:46.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.151384135000001, 2024-09-18T01:03:46.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147984651999998, 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:46.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147960091999998, 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:46.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147918574999998, 2024-09-18T01:03:46.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147898073, 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:46.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:46.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147877606999998, 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:46.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147836267000002, 2024-09-18T01:03:46.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:46.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:46.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147816229, 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:46.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:46.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147773897999997, 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:46.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147752906000001, 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:46.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.147701935999997, 2024-09-18T01:03:46.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:46.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.9155615479999999, 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:46.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:03:46.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:03:46.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:03:46.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:46.596 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:46.597 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:46.596+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:46.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3512 write 0xa15b000 thru 0xa16085e (0x585f bytes) 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3513 trunc from 0xa160a00 to 0xeaec000 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3514 write 0xdab5400 thru 0xdab822b (0x2e2c bytes) 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3515 write 0x21ac800 thru 0x21ba039 (0xd83a bytes) 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3517 trunc from 0xeaec000 to 0x687c200 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3523 write 0x3d7d000 thru 0x3d7d49b (0x49c bytes) 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3524 write 0x9151200 thru 0x915bf57 (0xad58 bytes) 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3527 read 0x5e4e00 thru 0x5ee48e (0x968f bytes) 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3528 read 0x141e200 thru 0x141e5a5 (0x3a6 bytes) 2024-09-18T01:03:46.599 INFO:teuthology.orchestra.run.smithi164.stdout:3529 trunc from 0x915c000 to 0x59bfc00 2024-09-18T01:03:46.615 INFO:teuthology.orchestra.run.smithi164.stdout:3530 punch from 0x4a48000 to 0x4a52e00, (0xae00 bytes) 2024-09-18T01:03:46.631 INFO:teuthology.orchestra.run.smithi164.stdout:3531 trunc from 0x59bfc00 to 0xd54bc00 2024-09-18T01:03:46.647 INFO:teuthology.orchestra.run.smithi164.stdout:3532 trunc from 0xd54bc00 to 0x6d51400 2024-09-18T01:03:46.658 INFO:teuthology.orchestra.run.smithi164.stdout:3533 trunc from 0x6d51400 to 0x7e8d600 2024-09-18T01:03:46.674 INFO:teuthology.orchestra.run.smithi164.stdout:3534 trunc from 0x7e8d600 to 0x8ff2400 2024-09-18T01:03:46.691 INFO:teuthology.orchestra.run.smithi164.stdout:3537 trunc from 0x8ff2400 to 0xa11e200 2024-09-18T01:03:46.698 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:46.697+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:46.701 INFO:teuthology.orchestra.run.smithi164.stdout:3539 read 0x8dbc600 thru 0x8dc5a2f (0x9430 bytes) 2024-09-18T01:03:46.702 INFO:teuthology.orchestra.run.smithi164.stdout:3540 write 0x28bbc00 thru 0x28c8545 (0xc946 bytes) 2024-09-18T01:03:46.702 INFO:teuthology.orchestra.run.smithi164.stdout:3543 read 0x9a53600 thru 0x9a5a0df (0x6ae0 bytes) 2024-09-18T01:03:46.721 INFO:teuthology.orchestra.run.smithi164.stdout:3544 write 0xe0fdc00 thru 0xe10b860 (0xdc61 bytes) 2024-09-18T01:03:46.728 INFO:teuthology.orchestra.run.smithi164.stdout:3545 clone 24 order 22 su 262144 sc 3 2024-09-18T01:03:46.753 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:46.754 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:46.762 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:46.807 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:46.806+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:46.869 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:46.869 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:46.869 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:46.869 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:46.869 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:46.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:46.919 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:46.918+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:47.002 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:47.001+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:47.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.290474332999999, 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.290454375000003, 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:47.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.290435434000003, 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:47.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:47.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:47.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.290413925000003, 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:47.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:47.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.290388196999999, 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:47.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281656337999998, 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:47.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:47.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:47.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281633624000001, 2024-09-18T01:03:47.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:47.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281614494000003, 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:47.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:47.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:47.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281595648, 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:47.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:47.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.086 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281576770000001, 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:47.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281554905, 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:47.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:47.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281531270000002, 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:47.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281497620000003, 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:47.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:47.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281469764999997, 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:47.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:47.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:47.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:47.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281449942000002, 2024-09-18T01:03:47.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:47.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281429508000002, 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:47.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281407428000001, 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:47.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281385106000002, 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:47.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:47.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:47.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281362710000003, 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:47.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.281342291000001, 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:47.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:47.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:47.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:47.155 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:47.156 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:47.157 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:47.155+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:47.257 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:47.256+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:47.262 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:47.262 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:47.262 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:47.270 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:47.293 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:03:47.357 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:47.356+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:47.364 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone23 from 0xe10ba00 (overlap 0x12b5c00) to 0x0 2024-09-18T01:03:47.426 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:47.426 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:47.427 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:47.435 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:47.458 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:47.457+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:47.559 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:47.558+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:47.584 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.538923984, 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.585 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:03:47.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.589 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.538912832999998, 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.591 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:03:47.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.591 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:47.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:47.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:03:47.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.595 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.538901002999999, 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.596 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:03:47.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:03:47.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.600 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.538884320000001, 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.601 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:47.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.538873082999999, 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:03:47.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:03:47.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:47.610 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.537550977999999, 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:03:47.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:03:47.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:47.615 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.537538263000002, 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:03:47.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:03:47.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:03:47.620 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.620 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.537493279000003, 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.621 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.537465654999998, 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.626 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:03:47.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.533918276000001, 2024-09-18T01:03:47.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:47.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:03:47.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.533558169000003, 2024-09-18T01:03:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:47.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:03:47.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:03:47.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.533532055999999, 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.640 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:03:47.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.644 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.531352667999997, 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:03:47.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:03:47.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:03:47.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:03:47.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.531342873, 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:03:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:03:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:03:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:03:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:03:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:03:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.531332094, 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:03:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:03:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.531312298000003, 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:03:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.529528778, 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:03:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.529501940000003, 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:03:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 45.529491391999997, 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:03:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 10.057564515999999, 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:03:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:47.691 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:47.693 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-18T01:03:47.693 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:47.692+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:47.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:47.705 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:47.705 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:47.705 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T01:03:47.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T01:03:47.793 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:47.792+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:47.855 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:47.855 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:47.855 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:47.855 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:47.865 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:47.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-18T01:03:47.906 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:47.905+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:47.995 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:47.994+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:48.006 INFO:teuthology.orchestra.run.smithi164.stdout:3546 punch from 0xddb4200 to 0xddc0600, (0xc400 bytes) 2024-09-18T01:03:48.008 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:48.009 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:48.009 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:48.009 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:48.009 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:48.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T01:03:48.018 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:48.019 INFO:teuthology.orchestra.run.smithi164.stdout:3547 punch from 0x85cf400 to 0x85d6e00, (0x7a00 bytes) 2024-09-18T01:03:48.033 INFO:teuthology.orchestra.run.smithi164.stdout:3549 trunc from 0xe10ba00 to 0x63dca00 2024-09-18T01:03:48.057 INFO:teuthology.orchestra.run.smithi164.stdout:3550 read 0x34ee400 thru 0x34fd335 (0xef36 bytes) 2024-09-18T01:03:48.062 INFO:teuthology.orchestra.run.smithi164.stdout:3551 trunc from 0x63dca00 to 0x2dbc800 2024-09-18T01:03:48.096 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:48.095+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:48.112 INFO:teuthology.orchestra.run.smithi164.stdout:3552 write 0x9b0cc00 thru 0x9b19124 (0xc525 bytes) 2024-09-18T01:03:48.112 INFO:teuthology.orchestra.run.smithi164.stdout:3555 punch from 0x8297800 to 0x82a7200, (0xfa00 bytes) 2024-09-18T01:03:48.123 INFO:teuthology.orchestra.run.smithi164.stdout:3556 trunc from 0x9b19200 to 0x1772400 2024-09-18T01:03:48.132 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.835272420000003, 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:03:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.835248303, 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:03:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.835169063000002, 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:03:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:03:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.835132844, 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:03:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.835104231000003, 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:03:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.835083365999999, 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:03:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.834465649000002, 2024-09-18T01:03:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:03:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.834436382, 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:03:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:03:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.169 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.834370090999997, 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:48.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:03:48.173 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.834249323999998, 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.174 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:48.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.834030878, 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:03:48.178 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.179 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.834009516999998, 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.183 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833855577999998, 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:03:48.187 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:03:48.189 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.190 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.191 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833835860000001, 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.192 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:03:48.193 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.194 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:03:48.195 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833815549999997, 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.196 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.197 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.198 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:03:48.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833795664, 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:03:48.201 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.202 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.203 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:03:48.204 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833775705000001, 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.205 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.207 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833755707000002, 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.210 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.211 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:03:48.212 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833736039999998, 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:03:48.214 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.216 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:03:48.217 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 41.833716138, 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:48.219 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.220 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:48.221 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:48.222 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:48.223 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:48.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T01:03:48.226 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:48.223+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:48.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3557 read 0x91f000 thru 0x929707 (0xa708 bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3558 read 0x2bc400 thru 0x2bc64e (0x24f bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3559 read 0x145c600 thru 0x1460bc7 (0x45c8 bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3560 trunc from 0x1772400 to 0x850c400 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3565 punch from 0x2ee200 to 0x2fb600, (0xd400 bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3566 write 0x5c0ee00 thru 0x5c1174d (0x294e bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3567 punch from 0x1992600 to 0x19a2200, (0xfc00 bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3570 write 0x1e7ac00 thru 0x1e7ca8d (0x1e8e bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3572 trunc from 0x850c400 to 0xa669000 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3573 read 0x2d8e00 thru 0x2e4c15 (0xbe16 bytes) 2024-09-18T01:03:48.228 INFO:teuthology.orchestra.run.smithi164.stdout:3574 read 0x587cc00 thru 0x5885b84 (0x8f85 bytes) 2024-09-18T01:03:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:3575 write 0x3a59800 thru 0x3a67595 (0xdd96 bytes) 2024-09-18T01:03:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:3577 write 0x2adc000 thru 0x2ae06d3 (0x46d4 bytes) 2024-09-18T01:03:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:3579 write 0x1fc3800 thru 0x1fcfacd (0xc2ce bytes) 2024-09-18T01:03:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:3580 punch from 0x8ea0600 to 0x8ea0c00, (0x600 bytes) 2024-09-18T01:03:48.229 INFO:teuthology.orchestra.run.smithi164.stdout:3584 punch from 0x561d600 to 0x5623000, (0x5a00 bytes) 2024-09-18T01:03:48.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:48.230 INFO:teuthology.orchestra.run.smithi164.stdout:3585 trunc from 0xa669000 to 0x3fab600 2024-09-18T01:03:48.260 INFO:teuthology.orchestra.run.smithi164.stdout:3586 write 0xbd17e00 thru 0xbd19281 (0x1482 bytes) 2024-09-18T01:03:48.261 INFO:teuthology.orchestra.run.smithi164.stdout:3587 read 0x13da400 thru 0x13e69f3 (0xc5f4 bytes) 2024-09-18T01:03:48.261 INFO:teuthology.orchestra.run.smithi164.stdout:3592 punch from 0x4968200 to 0x4970a00, (0x8800 bytes) 2024-09-18T01:03:48.271 INFO:teuthology.orchestra.run.smithi164.stdout:3594 trunc from 0xbd19400 to 0x43c1600 2024-09-18T01:03:48.284 INFO:teuthology.orchestra.run.smithi164.stdout:3595 clone 25 order 23 su 1048576 sc 10 2024-09-18T01:03:48.325 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:48.324+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:48.338 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:48.338 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:48.339 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:48.339 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:48.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:48.395 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:48.396 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:48.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-18T01:03:48.413 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:48.414 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:48.430 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:48.430 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:48.431 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:48.429+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:48.447 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:48.448 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:48.452 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:48.452 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:48.453 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:48.453 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:48.461 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:48.464 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:48.464 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:48.481 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:48.481 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:48.527 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:48.526+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:48.565 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:48.565 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:48.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:48.628 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:48.628+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:48.681 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:48.681 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:48.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:48.729 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:48.729+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:48.797 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:48.798 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:48.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:48.830 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:48.830+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:48.912 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:48.913 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:48.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:48.931 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:48.930+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:49.028 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.028 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.032 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:49.032+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:49.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:49.133 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:49.133+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:49.143 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.143 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:49.234 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:49.233+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:49.259 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.259 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:49.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:03:49.335 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:49.334+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:49.418 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.418 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:49.471 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:49.470+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.536 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:49.535+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:49.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:49.582 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone24 intact 2024-09-18T01:03:49.637 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:49.636+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:49.693 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.694 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.703 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:49.738 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:49.737+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:49.809 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.809 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.818 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:49.839 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:49.838+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:49.921 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:49.921 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:49.930 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:49.940 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-18T01:03:49.940 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:49.939+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:49.996 INFO:teuthology.orchestra.run.smithi164.stdout:3596 read 0x216d400 thru 0x2177b9a (0xa79b bytes) 2024-09-18T01:03:50.002 INFO:teuthology.orchestra.run.smithi164.stdout:3598 trunc from 0x43c1600 to 0x8283400 2024-09-18T01:03:50.015 INFO:teuthology.orchestra.run.smithi164.stdout:3599 write 0x1cfe600 thru 0x1d0c12c (0xdb2d bytes) 2024-09-18T01:03:50.016 INFO:teuthology.orchestra.run.smithi164.stdout:3601 read 0x801e800 thru 0x802074c (0x1f4d bytes) 2024-09-18T01:03:50.019 INFO:teuthology.orchestra.run.smithi164.stdout:3603 trunc from 0x8283400 to 0x51fd800 2024-09-18T01:03:50.038 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:50.038 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:50.041 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:50.040+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:50.048 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:50.051 INFO:teuthology.orchestra.run.smithi164.stdout:3605 read 0x2ae3a00 thru 0x2aec65b (0x8c5c bytes) 2024-09-18T01:03:50.056 INFO:teuthology.orchestra.run.smithi164.stdout:3606 punch from 0x16f7200 to 0x16fcc00, (0x5a00 bytes) 2024-09-18T01:03:50.057 INFO:teuthology.orchestra.run.smithi164.stdout:3612 read 0x20b4600 thru 0x20bc9df (0x83e0 bytes) 2024-09-18T01:03:50.067 INFO:teuthology.orchestra.run.smithi164.stdout:3614 trunc from 0x51fd800 to 0xc78f200 2024-09-18T01:03:50.080 INFO:teuthology.orchestra.run.smithi164.stdout:3615 write 0x4affa00 thru 0x4b0508c (0x568d bytes) 2024-09-18T01:03:50.090 INFO:teuthology.orchestra.run.smithi164.stdout:3618 trunc from 0xc78f200 to 0x4142400 2024-09-18T01:03:50.143 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:50.142+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:50.149 INFO:teuthology.orchestra.run.smithi164.stdout:3620 read 0x192ce00 thru 0x1932a9a (0x5c9b bytes) 2024-09-18T01:03:50.149 INFO:teuthology.orchestra.run.smithi164.stdout:3621 read 0x29a3c00 thru 0x29a7243 (0x3644 bytes) 2024-09-18T01:03:50.150 INFO:teuthology.orchestra.run.smithi164.stdout:3622 read 0x3d3b200 thru 0x3d3bc77 (0xa78 bytes) 2024-09-18T01:03:50.169 INFO:teuthology.orchestra.run.smithi164.stdout:3623 write 0xa956600 thru 0xa964058 (0xda59 bytes) 2024-09-18T01:03:50.170 INFO:teuthology.orchestra.run.smithi164.stdout:3624 trunc from 0xa964200 to 0xaa89200 2024-09-18T01:03:50.183 INFO:teuthology.orchestra.run.smithi164.stdout:3627 read 0x2fb0a00 thru 0x2fbc5c0 (0xbbc1 bytes) 2024-09-18T01:03:50.186 INFO:teuthology.orchestra.run.smithi164.stdout:3628 trunc from 0xaa89200 to 0x8d28c00 2024-09-18T01:03:50.201 INFO:teuthology.orchestra.run.smithi164.stdout:3629 write 0x1394600 thru 0x1399bd6 (0x55d7 bytes) 2024-09-18T01:03:50.201 INFO:teuthology.orchestra.run.smithi164.stdout:3631 read 0x7801800 thru 0x7804257 (0x2a58 bytes) 2024-09-18T01:03:50.201 INFO:teuthology.orchestra.run.smithi164.stdout:3634 read 0x8378000 thru 0x837b1aa (0x31ab bytes) 2024-09-18T01:03:50.202 INFO:teuthology.orchestra.run.smithi164.stdout:3637 trunc from 0x8d28c00 to 0x8b60c00 2024-09-18T01:03:50.208 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:50.208 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:50.217 INFO:teuthology.orchestra.run.smithi164.stdout:3638 punch from 0x3e6a00 to 0x3f6200, (0xf800 bytes) 2024-09-18T01:03:50.218 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:50.219 INFO:teuthology.orchestra.run.smithi164.stdout:3639 read 0x1402c00 thru 0x1410cab (0xe0ac bytes) 2024-09-18T01:03:50.220 INFO:teuthology.orchestra.run.smithi164.stdout:3643 trunc from 0x8b60c00 to 0x87d5400 2024-09-18T01:03:50.230 INFO:teuthology.orchestra.run.smithi164.stdout:3644 punch from 0x6666400 to 0x666d400, (0x7000 bytes) 2024-09-18T01:03:50.232 INFO:teuthology.orchestra.run.smithi164.stdout:3645 write 0x61d0200 thru 0x61dd3df (0xd1e0 bytes) 2024-09-18T01:03:50.232 INFO:teuthology.orchestra.run.smithi164.stdout:3646 read 0x83aba00 thru 0x83b49c6 (0x8fc7 bytes) 2024-09-18T01:03:50.234 INFO:teuthology.orchestra.run.smithi164.stdout:3647 trunc from 0x87d5400 to 0x706ea00 2024-09-18T01:03:50.244 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:50.243+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:50.247 INFO:teuthology.orchestra.run.smithi164.stdout:3648 read 0x4608000 thru 0x460fc9d (0x7c9e bytes) 2024-09-18T01:03:50.250 INFO:teuthology.orchestra.run.smithi164.stdout:3650 punch from 0x5544200 to 0x554ee00, (0xac00 bytes) 2024-09-18T01:03:50.254 INFO:teuthology.orchestra.run.smithi164.stdout:3651 trunc from 0x706ea00 to 0x36a0400 2024-09-18T01:03:50.272 INFO:teuthology.orchestra.run.smithi164.stdout:3652 punch from 0x752400 to 0x753000, (0xc00 bytes) 2024-09-18T01:03:50.277 INFO:teuthology.orchestra.run.smithi164.stdout:3654 write 0x10d0c00 thru 0x10d83a4 (0x77a5 bytes) 2024-09-18T01:03:50.277 INFO:teuthology.orchestra.run.smithi164.stdout:3659 read 0x4ab800 thru 0x4b0005 (0x4806 bytes) 2024-09-18T01:03:50.279 INFO:teuthology.orchestra.run.smithi164.stdout:3661 trunc from 0x36a0400 to 0x20cca00 2024-09-18T01:03:50.293 INFO:teuthology.orchestra.run.smithi164.stdout:3662 punch from 0x1e0d200 to 0x1e19a00, (0xc800 bytes) 2024-09-18T01:03:50.297 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:03:50.304 INFO:teuthology.orchestra.run.smithi164.stdout:3665 punch from 0x88f400 to 0x894800, (0x5400 bytes) 2024-09-18T01:03:50.304 INFO:teuthology.orchestra.run.smithi164.stdout:3667 write 0x12f5400 thru 0x12f72fa (0x1efb bytes) 2024-09-18T01:03:50.320 INFO:teuthology.orchestra.run.smithi164.stdout:3668 write 0x284c000 thru 0x2856383 (0xa384 bytes) 2024-09-18T01:03:50.343 INFO:teuthology.orchestra.run.smithi164.stdout:3669 write 0x732ae00 thru 0x732f7b6 (0x49b7 bytes) 2024-09-18T01:03:50.345 INFO:teuthology.orchestra.run.smithi164.stdout:3671 read 0x4d54000 thru 0x4d58810 (0x4811 bytes) 2024-09-18T01:03:50.354 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:50.353+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:50.360 INFO:teuthology.orchestra.run.smithi164.stdout:3672 write 0x99ff600 thru 0x9a0e469 (0xee6a bytes) 2024-09-18T01:03:50.361 INFO:teuthology.orchestra.run.smithi164.stdout:3673 write 0x8029400 thru 0x802a2bf (0xec0 bytes) 2024-09-18T01:03:50.365 INFO:teuthology.orchestra.run.smithi164.stdout:3675 trunc from 0x9a0e600 to 0x5b52200 2024-09-18T01:03:50.372 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:50.372 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:50.382 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:50.384 INFO:teuthology.orchestra.run.smithi164.stdout:3677 trunc from 0x5b52200 to 0x6440a00 2024-09-18T01:03:50.411 INFO:teuthology.orchestra.run.smithi164.stdout:3678 write 0xa19bc00 thru 0xa19e4d4 (0x28d5 bytes) 2024-09-18T01:03:50.418 INFO:teuthology.orchestra.run.smithi164.stdout:3679 write 0x8eb4e00 thru 0x8eba53c (0x573d bytes) 2024-09-18T01:03:50.418 INFO:teuthology.orchestra.run.smithi164.stdout:3680 read 0x766bc00 thru 0x76774cb (0xb8cc bytes) 2024-09-18T01:03:50.419 INFO:teuthology.orchestra.run.smithi164.stdout:3683 write 0x7de6600 thru 0x7df0aa4 (0xa4a5 bytes) 2024-09-18T01:03:50.419 INFO:teuthology.orchestra.run.smithi164.stdout:3686 punch from 0x178dc00 to 0x1797800, (0x9c00 bytes) 2024-09-18T01:03:50.424 INFO:teuthology.orchestra.run.smithi164.stdout:3687 trunc from 0xa19e600 to 0xdc52600 2024-09-18T01:03:50.434 INFO:teuthology.orchestra.run.smithi164.stdout:3688 read 0x155de00 thru 0x156a966 (0xcb67 bytes) 2024-09-18T01:03:50.442 INFO:teuthology.orchestra.run.smithi164.stdout:3689 write 0x4cba200 thru 0x4cc3a9f (0x98a0 bytes) 2024-09-18T01:03:50.442 INFO:teuthology.orchestra.run.smithi164.stdout:3693 punch from 0x5144600 to 0x5148a00, (0x4400 bytes) 2024-09-18T01:03:50.445 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:50.444+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:50.450 INFO:teuthology.orchestra.run.smithi164.stdout:3695 trunc from 0xdc52600 to 0x7147800 2024-09-18T01:03:50.468 INFO:teuthology.orchestra.run.smithi164.stdout:3701 punch from 0x3626600 to 0x3631200, (0xac00 bytes) 2024-09-18T01:03:50.473 INFO:teuthology.orchestra.run.smithi164.stdout:3702 read 0x609aa00 thru 0x609d3e7 (0x29e8 bytes) 2024-09-18T01:03:50.474 INFO:teuthology.orchestra.run.smithi164.stdout:3704 write 0x4ad0a00 thru 0x4ad59a8 (0x4fa9 bytes) 2024-09-18T01:03:50.475 INFO:teuthology.orchestra.run.smithi164.stdout:3705 write 0x6d70400 thru 0x6d742d5 (0x3ed6 bytes) 2024-09-18T01:03:50.476 INFO:teuthology.orchestra.run.smithi164.stdout:3706 trunc from 0x7147800 to 0x7b97200 2024-09-18T01:03:50.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:50.495 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:50.495 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:50.504 INFO:teuthology.orchestra.run.smithi164.stdout:3707 write 0xbbbdc00 thru 0xbbc4c96 (0x7097 bytes) 2024-09-18T01:03:50.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:50.510 INFO:teuthology.orchestra.run.smithi164.stdout:3711 clone 26 order 22 su 65536 sc 14 2024-09-18T01:03:50.546 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:50.545+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:50.618 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:50.649 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:50.649 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:50.651 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:50.650+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:50.653 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:50.657 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:51.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:51.055 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:51.054+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:51.056 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:51.056 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:51.056 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:51.156 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:51.155+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:51.158 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:51.158 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:51.158 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:51.158 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:51.158 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:51.167 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:51.257 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:51.256+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:51.313 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 78.937367287000001, 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.314 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:51.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:51.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.318 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.939355981999995, 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.321 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.322 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:51.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:51.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:51.327 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.931679029999998, 2024-09-18T01:03:51.328 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.329 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.330 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:51.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.332 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.333 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:51.334 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:51.335 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 67.833821080000007, 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.336 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.337 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:51.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.339 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.340 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.341 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.977740068000003, 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.342 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.344 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.345 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:51.346 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:51.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.348 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.977370772, 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.349 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:51.350 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.351 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:51.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.353 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.354 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.975631927000002, 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:51.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:51.357 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.975560037000001, 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:51.361 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.362 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:51.363 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.887694027000002, 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.364 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.366 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.367 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.368 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:51.369 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.881502767999997, 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:51.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:51.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.879973083000003, 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.375 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:51.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:51.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.380 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.879593393999997, 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.381 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:51.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:51.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.864094991000002, 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.387 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:51.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:51.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.393 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.863888232000001, 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:51.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:51.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.863470475, 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:51.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:51.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:51.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:51.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.863261237000003, 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.405 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:51.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:51.409 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.863197303, 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:51.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:51.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.862449097999999, 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:51.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:51.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.603510292999999, 2024-09-18T01:03:51.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:51.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:51.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.425 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.602646675000003, 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:51.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:51.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:51.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.431 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.432 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.432 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:51.432 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:51.432 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:51.433 INFO:teuthology.orchestra.run.smithi164.stdout:flattening image image_client.0-clone25 2024-09-18T01:03:51.433 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:51.432+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:51.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:51.533 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:51.532+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:51.547 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:51.547 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:51.548 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:51.548 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:51.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:51.635 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:51.634+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:51.712 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:51.712 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:51.712 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:51.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:51.764 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:51.763+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:51.778 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-18T01:03:51.836 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:51.835+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:51.840 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.519485639999999, 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:03:51.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.519465130999997, 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:03:51.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:03:51.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.519445326000003, 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:03:51.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:03:51.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.519410686000001, 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:03:51.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.519389185000001, 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:03:51.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.519366007999999, 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:03:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:03:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515377176000001, 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:03:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:03:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515357623, 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:03:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515337119000002, 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:03:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515316255000002, 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:03:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515296354999997, 2024-09-18T01:03:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:03:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:03:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515272267, 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:03:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:03:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515240947000002, 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:03:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:03:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515196340999999, 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:03:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515165549999999, 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:03:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515142064999999, 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:03:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.51511301, 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:03:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:03:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.515092250999999, 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:03:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:03:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2571515039999994, 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:03:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2570677949999993, 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:03:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:03:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:03:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:51.938 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:51.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:51.940 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:51.938+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:52.028 INFO:teuthology.orchestra.run.smithi164.stdout:3713 write 0xb2f3200 thru 0xb2fb5a0 (0x83a1 bytes) 2024-09-18T01:03:52.029 INFO:teuthology.orchestra.run.smithi164.stdout:3714 read 0xbb71c00 thru 0xbb73e4b (0x224c bytes) 2024-09-18T01:03:52.031 INFO:teuthology.orchestra.run.smithi164.stdout:3715 trunc from 0xbbc4e00 to 0xa213a00 2024-09-18T01:03:52.040 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:52.039+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:52.087 INFO:teuthology.orchestra.run.smithi164.stdout:3716 read 0x54de400 thru 0x54e29ff (0x4600 bytes) 2024-09-18T01:03:52.093 INFO:teuthology.orchestra.run.smithi164.stdout:3717 trunc from 0xa213a00 to 0x63ffa00 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:993 3.4 3.6cf1c604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105848+0000", 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00047835500000000002, 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050048099999999999, 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 993 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105848+0000", 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105848+0000", 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105849+0000", 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105862+0000", 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3661e-05 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105869+0000", 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8419999999999999e-06 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105876+0000", 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:994 3.4 3.6cf1c604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105885+0000", 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00044154799999999999, 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052829099999999996, 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 994 2024-09-18T01:03:52.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105885+0000", 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105885+0000", 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105886+0000", 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000001e-07 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105893+0000", 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1810000000000004e-06 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105900+0000", 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8850000000000001e-06 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105905+0000", 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1010 3.7 3.52eb2c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105912+0000", 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000414497, 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056094300000000005, 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1010 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105912+0000", 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105912+0000", 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105913+0000", 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105915+0000", 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0200000000000001e-06 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105923+0000", 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9030000000000004e-06 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105928+0000", 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7930000000000003e-06 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.122 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:923 3.10 3.6ff4110 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105067+0000", 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0012598659999999999, 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014531749999999999, 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 923 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105067+0000", 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105067+0000", 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105067+0000", 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1600000000000002e-07 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105069+0000", 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9199999999999998e-06 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105076+0000", 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1450000000000002e-06 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105081+0000", 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1011 3.7 3.52eb2c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105936+0000", 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00039085699999999999, 2024-09-18T01:03:52.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00062780500000000005, 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1011 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105936+0000", 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105936+0000", 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105936+0000", 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-18T01:03:52.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105938+0000", 2024-09-18T01:03:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1050000000000002e-06 2024-09-18T01:03:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105945+0000", 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4459999999999998e-06 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105950+0000", 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2530000000000004e-06 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:924 3.10 3.6ff4110 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105090+0000", 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.001236713, 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001508969, 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 924 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105090+0000", 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105090+0000", 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105090+0000", 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105097+0000", 2024-09-18T01:03:52.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8739999999999997e-06 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105104+0000", 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0049999999999998e-06 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105109+0000", 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1012 3.7 3.52eb2c7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105957+0000", 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00036985799999999998, 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068678200000000004, 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1012 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105957+0000", 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105957+0000", 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105957+0000", 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2900000000000004e-07 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105963+0000", 2024-09-18T01:03:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5659999999999997e-06 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105970+0000", 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2559999999999996e-06 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105976+0000", 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:928 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105118+0000", 2024-09-18T01:03:52.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.001209101, 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001558315, 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 928 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105118+0000", 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105118+0000", 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105118+0000", 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6899999999999998e-07 2024-09-18T01:03:52.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105120+0000", 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1509999999999998e-06 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105128+0000", 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9780000000000005e-06 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105134+0000", 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7989999999999999e-06 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1032 3.10 3.6ff4110 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105983+0000", 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00034389499999999999, 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074371599999999995, 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1032 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105983+0000", 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105983+0000", 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105983+0000", 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0699999999999997e-07 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105986+0000", 2024-09-18T01:03:52.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2929999999999999e-06 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105993+0000", 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.917e-06 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105998+0000", 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9840000000000001e-06 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:929 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105142+0000", 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0011842140000000001, 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016480049999999999, 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 929 2024-09-18T01:03:52.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105142+0000", 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105142+0000", 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105143+0000", 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105145+0000", 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105152+0000", 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0670000000000003e-06 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105158+0000", 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.468e-06 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1033 3.10 3.6ff4110 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106005+0000", 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00032170000000000001, 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084197799999999995, 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1033 2024-09-18T01:03:52.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106005+0000", 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106005+0000", 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106005+0000", 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106008+0000", 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2199999999999999e-06 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106014+0000", 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-18T01:03:52.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106019+0000", 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:930 3.0 3.de399480 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.155 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105165+0000", 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.001161593, 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017390960000000001, 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 930 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105165+0000", 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105165+0000", 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105166+0000", 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8500000000000001e-07 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105168+0000", 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1710000000000001e-06 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105174+0000", 2024-09-18T01:03:52.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6939999999999998e-06 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105180+0000", 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.852e-06 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1036 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106026+0000", 2024-09-18T01:03:52.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00030018699999999999, 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093329299999999997, 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1036 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106026+0000", 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106026+0000", 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106027+0000", 2024-09-18T01:03:52.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999998e-07 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106029+0000", 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106036+0000", 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-18T01:03:52.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106041+0000", 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:931 3.0 3.de399480 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.163 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105191+0000", 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0011353730000000001, 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018261289999999999, 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 931 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105191+0000", 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105191+0000", 2024-09-18T01:03:52.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105192+0000", 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1099999999999996e-07 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105195+0000", 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105202+0000", 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105210+0000", 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7449999999999995e-06 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1037 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106048+0000", 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00027874799999999999, 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001025911, 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1037 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106048+0000", 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106048+0000", 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106048+0000", 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.58e-07 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106051+0000", 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106067+0000", 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6421000000000001e-05 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106072+0000", 2024-09-18T01:03:52.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1038 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106088+0000", 2024-09-18T01:03:52.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000238595, 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001040242, 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1038 2024-09-18T01:03:52.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106088+0000", 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106088+0000", 2024-09-18T01:03:52.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106089+0000", 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106091+0000", 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5550000000000001e-06 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106100+0000", 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0790000000000007e-06 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106106+0000", 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1039 3.0 3.de399480 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106122+0000", 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00020492000000000001, 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001061088, 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1039 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106122+0000", 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106122+0000", 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106122+0000", 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106125+0000", 2024-09-18T01:03:52.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.554e-06 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106131+0000", 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8850000000000004e-06 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106136+0000", 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5489999999999997e-06 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1040 3.0 3.de399480 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106154+0000", 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00017305900000000001, 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00107413, 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1040 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106154+0000", 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106154+0000", 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106154+0000", 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7000000000000005e-07 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106157+0000", 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.779e-06 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106164+0000", 2024-09-18T01:03:52.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106170+0000", 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7729999999999996e-06 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1041 3.2 3:43e3013b:::rbd_data.11039e8b54c5.0000000000000006:1c [sparse-read 5439488~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106180+0000", 2024-09-18T01:03:52.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000146611, 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001078795, 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1041 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106180+0000", 2024-09-18T01:03:52.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106180+0000", 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106180+0000", 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106183+0000", 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4439999999999998e-06 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106189+0000", 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3199999999999996e-06 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106196+0000", 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6209999999999998e-06 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.107131+0000", 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093494599999999998 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.107200+0000", 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9046000000000003e-05 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1042 3.2 3.dc80c7c2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106204+0000", 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000123029, 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001097974, 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1042 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.189 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106204+0000", 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106204+0000", 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106204+0000", 2024-09-18T01:03:52.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.75e-07 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106212+0000", 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106219+0000", 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7989999999999996e-06 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106226+0000", 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0169999999999999e-06 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1043 3.2 3.dc80c7c2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106235+0000", 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2101999999999993e-05, 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011167729999999999, 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1043 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106235+0000", 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106235+0000", 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106235+0000", 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7e-07 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106237+0000", 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.384e-06 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106245+0000", 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9100000000000005e-06 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106250+0000", 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9840000000000001e-06 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:1044 3.4 3.6cf1c604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.106259+0000", 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.7970000000000001e-05, 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001147725, 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1044 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106259+0000", 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106259+0000", 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106259+0000", 2024-09-18T01:03:52.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106262+0000", 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106268+0000", 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3230000000000003e-06 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106282+0000", 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3715e-05 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.199 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:958 3.7 3:e34d74a0:::rbd_data.11039e8b54c5.000000000000000d:1c [sparse-read 2424832~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105492+0000", 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00083491900000000005, 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019731699999999998, 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 958 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105492+0000", 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105492+0000", 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105492+0000", 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2899999999999999e-07 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105494+0000", 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2639999999999998e-06 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105501+0000", 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.567e-06 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105506+0000", 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0980000000000001e-06 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.106337+0000", 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000830998 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.107349+0000", 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010114950000000001 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:979 3.10 3.6ff4110 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105513+0000", 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00081369999999999999, 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020189370000000002, 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 979 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105513+0000", 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105513+0000", 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105513+0000", 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105516+0000", 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2290000000000002e-06 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105523+0000", 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.593e-06 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105529+0000", 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:980 3.10 3.6ff4110 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105536+0000", 2024-09-18T01:03:52.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000790502, 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020500000000000002, 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 980 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105536+0000", 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105536+0000", 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105536+0000", 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105539+0000", 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0540000000000002e-06 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105545+0000", 2024-09-18T01:03:52.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.314e-06 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105550+0000", 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3890000000000004e-06 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:983 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105562+0000", 2024-09-18T01:03:52.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00076439100000000001, 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002078838, 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 983 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105562+0000", 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105562+0000", 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105563+0000", 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105565+0000", 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.491e-06 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105572+0000", 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7009999999999999e-06 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105578+0000", 2024-09-18T01:03:52.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:984 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105586+0000", 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00074092699999999995, 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021111749999999999, 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 984 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105586+0000", 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105586+0000", 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105586+0000", 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105588+0000", 2024-09-18T01:03:52.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2460000000000002e-06 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105595+0000", 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3029999999999996e-06 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105600+0000", 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:985 3.0 3.a314ed40 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105607+0000", 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00071940199999999995, 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021443009999999999, 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 985 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105607+0000", 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105607+0000", 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105608+0000", 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105610+0000", 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.221e-06 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105616+0000", 2024-09-18T01:03:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105632+0000", 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5449999999999999e-05 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:986 3.0 3.de399480 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105646+0000", 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00068094300000000004, 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021573180000000001, 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 986 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105646+0000", 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105646+0000", 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105646+0000", 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1799999999999995e-07 2024-09-18T01:03:52.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105649+0000", 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7209999999999999e-06 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105655+0000", 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3720000000000001e-06 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105661+0000", 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2989999999999996e-06 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5234.0:987 3.0 3.de399480 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:52.105669+0000", 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00065734500000000004, 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002189084, 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5234", 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/33706765", 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 987 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105669+0000", 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105669+0000", 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105670+0000", 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105672+0000", 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2510000000000001e-06 2024-09-18T01:03:52.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105688+0000", 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5644000000000001e-05 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:52.105699+0000", 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0966e-05 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.232 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:52.232 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 30 2024-09-18T01:03:52.232 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:52.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:52.235 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:52.232+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:52.236 INFO:teuthology.orchestra.run.smithi164.stdout:3720 read 0x1560400 thru 0x156b1cc (0xadcd bytes) 2024-09-18T01:03:52.236 INFO:teuthology.orchestra.run.smithi164.stdout:3722 read 0x3845200 thru 0x38461e3 (0xfe4 bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3723 trunc from 0x63ffa00 to 0x75c4c00 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3724 write 0x31da00 thru 0x31dd2a (0x32b bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3725 punch from 0x33e0600 to 0x33e7600, (0x7000 bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3726 read 0x4916600 thru 0x491f3b9 (0x8dba bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3727 write 0x3defc00 thru 0x3df07f5 (0xbf6 bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3728 write 0x964a00 thru 0x96fb00 (0xb101 bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3731 write 0x9f57e00 thru 0x9f61896 (0x9a97 bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3734 write 0xacca800 thru 0xacd85d8 (0xddd9 bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3735 write 0x12c8200 thru 0x12d2aab (0xa8ac bytes) 2024-09-18T01:03:52.237 INFO:teuthology.orchestra.run.smithi164.stdout:3737 write 0x8bac000 thru 0x8bb77a7 (0xb7a8 bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3739 write 0x5f3f000 thru 0x5f4591a (0x691b bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3742 read 0xa96ba00 thru 0xa96c651 (0xc52 bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3743 write 0xa544800 thru 0xa550e6f (0xc670 bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3748 read 0x1233000 thru 0x123fa73 (0xca74 bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3749 write 0x762e400 thru 0x7638369 (0x9f6a bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3750 read 0x46a6400 thru 0x46ac4f1 (0x60f2 bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3751 write 0x563ec00 thru 0x5645cf9 (0x70fa bytes) 2024-09-18T01:03:52.238 INFO:teuthology.orchestra.run.smithi164.stdout:3752 read 0x216a600 thru 0x2170353 (0x5d54 bytes) 2024-09-18T01:03:52.243 INFO:teuthology.orchestra.run.smithi164.stdout:3755 trunc from 0xacd8600 to 0x1021400 2024-09-18T01:03:52.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:03:52.334 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:52.333+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:52.363 INFO:teuthology.orchestra.run.smithi164.stdout:3757 read 0x962400 thru 0x967399 (0x4f9a bytes) 2024-09-18T01:03:52.365 INFO:teuthology.orchestra.run.smithi164.stdout:3758 punch from 0xbd6800 to 0xbdcc00, (0x6400 bytes) 2024-09-18T01:03:52.380 INFO:teuthology.orchestra.run.smithi164.stdout:3759 trunc from 0x1021400 to 0xd4c2400 2024-09-18T01:03:52.389 INFO:teuthology.orchestra.run.smithi164.stdout:3760 write 0x4f3200 thru 0x4fe638 (0xb439 bytes) 2024-09-18T01:03:52.391 INFO:teuthology.orchestra.run.smithi164.stdout:3762 write 0x544d000 thru 0x5453c6e (0x6c6f bytes) 2024-09-18T01:03:52.391 INFO:teuthology.orchestra.run.smithi164.stdout:3766 read 0x1bfec00 thru 0x1c04130 (0x5531 bytes) 2024-09-18T01:03:52.400 INFO:teuthology.orchestra.run.smithi164.stdout:3767 trunc from 0xd4c2400 to 0x5bf3000 2024-09-18T01:03:52.418 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:52.418 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:52.418 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:52.418 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:52.418 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:52.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:52.434 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:52.433+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:52.445 INFO:teuthology.orchestra.run.smithi164.stdout:3769 write 0xe64ac00 thru 0xe64ed2c (0x412d bytes) 2024-09-18T01:03:52.446 INFO:teuthology.orchestra.run.smithi164.stdout:3771 read 0xd871400 thru 0xd871595 (0x196 bytes) 2024-09-18T01:03:52.446 INFO:teuthology.orchestra.run.smithi164.stdout:3772 write 0xbb1f600 thru 0xbb2da5c (0xe45d bytes) 2024-09-18T01:03:52.446 INFO:teuthology.orchestra.run.smithi164.stdout:3773 punch from 0x2d78800 to 0x2d81800, (0x9000 bytes) 2024-09-18T01:03:52.448 INFO:teuthology.orchestra.run.smithi164.stdout:3776 write 0x4f9b000 thru 0x4fa8b76 (0xdb77 bytes) 2024-09-18T01:03:52.448 INFO:teuthology.orchestra.run.smithi164.stdout:3777 punch from 0xc9bfc00 to 0xc9c2c00, (0x3000 bytes) 2024-09-18T01:03:52.449 INFO:teuthology.orchestra.run.smithi164.stdout:3779 write 0x9490400 thru 0x9490db9 (0x9ba bytes) 2024-09-18T01:03:52.449 INFO:teuthology.orchestra.run.smithi164.stdout:3783 write 0x3d17600 thru 0x3d1e8ba (0x72bb bytes) 2024-09-18T01:03:52.449 INFO:teuthology.orchestra.run.smithi164.stdout:3784 punch from 0x1451c00 to 0x145da00, (0xbe00 bytes) 2024-09-18T01:03:52.449 INFO:teuthology.orchestra.run.smithi164.stdout:3785 read 0xb5e2000 thru 0xb5ee787 (0xc788 bytes) 2024-09-18T01:03:52.449 INFO:teuthology.orchestra.run.smithi164.stdout:3787 read 0xc69f400 thru 0xc6a9bdd (0xa7de bytes) 2024-09-18T01:03:52.449 INFO:teuthology.orchestra.run.smithi164.stdout:3788 punch from 0x22ba200 to 0x22baa00, (0x800 bytes) 2024-09-18T01:03:52.449 INFO:teuthology.orchestra.run.smithi164.stdout:3790 read 0x1c35c00 thru 0x1c3d75e (0x7b5f bytes) 2024-09-18T01:03:52.456 INFO:teuthology.orchestra.run.smithi164.stdout:3794 trunc from 0xe64ee00 to 0xa01b600 2024-09-18T01:03:52.480 INFO:teuthology.orchestra.run.smithi164.stdout:3796 read 0x3a03000 thru 0x3a0fcfd (0xccfe bytes) 2024-09-18T01:03:52.481 INFO:teuthology.orchestra.run.smithi164.stdout:3798 write 0xf28600 thru 0xf2b52f (0x2f30 bytes) 2024-09-18T01:03:52.481 INFO:teuthology.orchestra.run.smithi164.stdout:3801 read 0x3df9c00 thru 0x3dfac6b (0x106c bytes) 2024-09-18T01:03:52.504 INFO:teuthology.orchestra.run.smithi164.stdout:3803 write 0xe343400 thru 0xe3497f8 (0x63f9 bytes) 2024-09-18T01:03:52.504 INFO:teuthology.orchestra.run.smithi164.stdout:3804 read 0xc42b800 thru 0xc4327db (0x6fdc bytes) 2024-09-18T01:03:52.504 INFO:teuthology.orchestra.run.smithi164.stdout:3808 read 0x9e5f800 thru 0x9e6465d (0x4e5e bytes) 2024-09-18T01:03:52.505 INFO:teuthology.orchestra.run.smithi164.stdout:3809 write 0x655400 thru 0x6636af (0xe2b0 bytes) 2024-09-18T01:03:52.506 INFO:teuthology.orchestra.run.smithi164.stdout:3810 read 0xb706c00 thru 0xb713a83 (0xce84 bytes) 2024-09-18T01:03:52.506 INFO:teuthology.orchestra.run.smithi164.stdout:3811 read 0xc0ca400 thru 0xc0cdc94 (0x3895 bytes) 2024-09-18T01:03:52.506 INFO:teuthology.orchestra.run.smithi164.stdout:3812 punch from 0xdbbf000 to 0xdbcc400, (0xd400 bytes) 2024-09-18T01:03:52.507 INFO:teuthology.orchestra.run.smithi164.stdout:3816 trunc from 0xe349800 to 0xec41c00 2024-09-18T01:03:52.535 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:52.534+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:52.543 INFO:teuthology.orchestra.run.smithi164.stdout:3817 trunc from 0xec41c00 to 0x4070600 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.276573806000002, 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.271947902999997, 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.271869854000002, 2024-09-18T01:03:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:52.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:52.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.215073811000003, 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:52.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.215047947000002, 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.215025466999997, 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.215004430999997, 2024-09-18T01:03:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:52.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:52.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:52.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:52.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:52.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:52.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:52.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:52.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:52.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.214977036999997, 2024-09-18T01:03:52.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:52.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:52.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.619 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:52.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:52.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:52.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:52.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:52.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:52.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.214952732, 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211553248999998, 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:52.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:52.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211528688999998, 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.640 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211487171999998, 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:52.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:52.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.21146667, 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:52.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211446203999998, 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:52.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211404864000002, 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:52.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:52.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211384826, 2024-09-18T01:03:52.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:52.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211342494999997, 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:52.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:52.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211321503000001, 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:52.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.211270532999997, 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:52.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.9791301449999992, 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:52.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:52.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:03:52.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:52.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:03:52.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:52.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:52.683 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:52.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:52.686 INFO:teuthology.orchestra.run.smithi164.stdout:3819 read 0x3c21400 thru 0x3c2c15f (0xad60 bytes) 2024-09-18T01:03:52.686 INFO:teuthology.orchestra.run.smithi164.stdout:3822 write 0x5993000 thru 0x59a099d (0xd99e bytes) 2024-09-18T01:03:52.686 INFO:teuthology.orchestra.run.smithi164.stdout:3825 write 0xa726400 thru 0xa72cebd (0x6abe bytes) 2024-09-18T01:03:52.686 INFO:teuthology.orchestra.run.smithi164.stdout:3826 write 0x262ec00 thru 0x263b6b7 (0xcab8 bytes) 2024-09-18T01:03:52.686 INFO:teuthology.orchestra.run.smithi164.stdout:3829 write 0x4760a00 thru 0x4763223 (0x2824 bytes) 2024-09-18T01:03:52.686 INFO:teuthology.orchestra.run.smithi164.stdout:3831 punch from 0x5909800 to 0x5910600, (0x6e00 bytes) 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3832 punch from 0x9618400 to 0x9620800, (0x8400 bytes) 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3833 trunc from 0xa72d000 to 0xb2d0a00 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3834 write 0x70ff000 thru 0x7103fbc (0x4fbd bytes) 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3836 write 0xcb42c00 thru 0xcb49a0c (0x6e0d bytes) 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3837 write 0xc561a00 thru 0xc568889 (0x6e8a bytes) 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3839 write 0x8ad7a00 thru 0x8ae36cb (0xbccc bytes) 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3842 punch from 0x545f800 to 0x5461a00, (0x2200 bytes) 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3843 trunc from 0xcb49c00 to 0xd4f0a00 2024-09-18T01:03:52.687 INFO:teuthology.orchestra.run.smithi164.stdout:3844 read 0xc566800 thru 0xc571772 (0xaf73 bytes) 2024-09-18T01:03:52.687 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:52.683+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:52.691 INFO:teuthology.orchestra.run.smithi164.stdout:3845 trunc from 0xd4f0a00 to 0x8982000 2024-09-18T01:03:52.718 INFO:teuthology.orchestra.run.smithi164.stdout:3846 trunc from 0x8982000 to 0xf6b000 2024-09-18T01:03:52.771 INFO:teuthology.orchestra.run.smithi164.stdout:3847 write 0xd5ef000 thru 0xd5fe049 (0xf04a bytes) 2024-09-18T01:03:52.773 INFO:teuthology.orchestra.run.smithi164.stdout:3848 write 0xce63400 thru 0xce698ee (0x64ef bytes) 2024-09-18T01:03:52.773 INFO:teuthology.orchestra.run.smithi164.stdout:3850 write 0x500000 thru 0x503b92 (0x3b93 bytes) 2024-09-18T01:03:52.773 INFO:teuthology.orchestra.run.smithi164.stdout:3851 read 0x2ec7a00 thru 0x2ed08f8 (0x8ef9 bytes) 2024-09-18T01:03:52.774 INFO:teuthology.orchestra.run.smithi164.stdout:3853 read 0xc007600 thru 0xc010105 (0x8b06 bytes) 2024-09-18T01:03:52.774 INFO:teuthology.orchestra.run.smithi164.stdout:3856 punch from 0x442fe00 to 0x443d200, (0xd400 bytes) 2024-09-18T01:03:52.774 INFO:teuthology.orchestra.run.smithi164.stdout:3857 punch from 0x1401600 to 0x1408800, (0x7200 bytes) 2024-09-18T01:03:52.775 INFO:teuthology.orchestra.run.smithi164.stdout:3861 read 0x867e600 thru 0x868340d (0x4e0e bytes) 2024-09-18T01:03:52.775 INFO:teuthology.orchestra.run.smithi164.stdout:3864 read 0x5120400 thru 0x512c80f (0xc410 bytes) 2024-09-18T01:03:52.775 INFO:teuthology.orchestra.run.smithi164.stdout:3865 punch from 0x7f03200 to 0x7f0ba00, (0x8800 bytes) 2024-09-18T01:03:52.775 INFO:teuthology.orchestra.run.smithi164.stdout:3866 write 0x7983e00 thru 0x79879fe (0x3bff bytes) 2024-09-18T01:03:52.775 INFO:teuthology.orchestra.run.smithi164.stdout:3867 read 0x1e2c00 thru 0x1f1564 (0xe965 bytes) 2024-09-18T01:03:52.776 INFO:teuthology.orchestra.run.smithi164.stdout:3870 write 0x7dff400 thru 0x7e0c62e (0xd22f bytes) 2024-09-18T01:03:52.776 INFO:teuthology.orchestra.run.smithi164.stdout:3872 read 0x52bfc00 thru 0x52cb6d9 (0xbada bytes) 2024-09-18T01:03:52.776 INFO:teuthology.orchestra.run.smithi164.stdout:3877 read 0xc291800 thru 0xc29407f (0x2880 bytes) 2024-09-18T01:03:52.776 INFO:teuthology.orchestra.run.smithi164.stdout:3881 write 0x5379800 thru 0x53810e9 (0x78ea bytes) 2024-09-18T01:03:52.778 INFO:teuthology.orchestra.run.smithi164.stdout:3882 trunc from 0xd5fe200 to 0xbc74000 2024-09-18T01:03:52.784 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:52.783+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:52.800 INFO:teuthology.orchestra.run.smithi164.stdout:3883 read 0x17abe00 thru 0x17b2677 (0x6878 bytes) 2024-09-18T01:03:52.803 INFO:teuthology.orchestra.run.smithi164.stdout:3884 punch from 0x4de8400 to 0x4df0e00, (0x8a00 bytes) 2024-09-18T01:03:52.803 INFO:teuthology.orchestra.run.smithi164.stdout:3886 write 0x34c9600 thru 0x34d0a10 (0x7411 bytes) 2024-09-18T01:03:52.804 INFO:teuthology.orchestra.run.smithi164.stdout:3887 read 0x4dd3200 thru 0x4ddc004 (0x8e05 bytes) 2024-09-18T01:03:52.805 INFO:teuthology.orchestra.run.smithi164.stdout:3889 read 0x4af5a00 thru 0x4afe23b (0x883c bytes) 2024-09-18T01:03:52.805 INFO:teuthology.orchestra.run.smithi164.stdout:3892 read 0x8449200 thru 0x844c5cc (0x33cd bytes) 2024-09-18T01:03:52.805 INFO:teuthology.orchestra.run.smithi164.stdout:3894 read 0x9764400 thru 0x9772beb (0xe7ec bytes) 2024-09-18T01:03:52.806 INFO:teuthology.orchestra.run.smithi164.stdout:3895 read 0x7ed8a00 thru 0x7ee2ac4 (0xa0c5 bytes) 2024-09-18T01:03:52.806 INFO:teuthology.orchestra.run.smithi164.stdout:3898 read 0x39bd600 thru 0x39cb7c9 (0xe1ca bytes) 2024-09-18T01:03:52.810 INFO:teuthology.orchestra.run.smithi164.stdout:3901 read 0x2ce8e00 thru 0x2cf6cbf (0xdec0 bytes) 2024-09-18T01:03:52.812 INFO:teuthology.orchestra.run.smithi164.stdout:3904 trunc from 0xbc74000 to 0xa940600 2024-09-18T01:03:52.825 INFO:teuthology.orchestra.run.smithi164.stdout:3905 read 0x9662200 thru 0x966d31a (0xb11b bytes) 2024-09-18T01:03:52.837 INFO:teuthology.orchestra.run.smithi164.stdout:3907 write 0xd5b1200 thru 0xd5bb882 (0xa683 bytes) 2024-09-18T01:03:52.838 INFO:teuthology.orchestra.run.smithi164.stdout:3911 write 0xcf800 thru 0xd1cb8 (0x24b9 bytes) 2024-09-18T01:03:52.839 INFO:teuthology.orchestra.run.smithi164.stdout:3914 write 0xa1f1c00 thru 0xa1f6016 (0x4417 bytes) 2024-09-18T01:03:52.839 INFO:teuthology.orchestra.run.smithi164.stdout:3918 write 0xbe12a00 thru 0xbe19c3c (0x723d bytes) 2024-09-18T01:03:52.839 INFO:teuthology.orchestra.run.smithi164.stdout:3920 punch from 0x6ba3000 to 0x6bad800, (0xa800 bytes) 2024-09-18T01:03:52.845 INFO:teuthology.orchestra.run.smithi164.stdout:3921 trunc from 0xd5bba00 to 0x82f2a00 2024-09-18T01:03:52.849 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:52.850 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:52.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:52.872 INFO:teuthology.orchestra.run.smithi164.stdout:3925 write 0x2dc400 thru 0x2e70af (0xacb0 bytes) 2024-09-18T01:03:52.887 INFO:teuthology.orchestra.run.smithi164.stdout:3926 write 0xb330200 thru 0xb333b25 (0x3926 bytes) 2024-09-18T01:03:52.893 INFO:teuthology.orchestra.run.smithi164.stdout:3930 write 0x971e800 thru 0x9727756 (0x8f57 bytes) 2024-09-18T01:03:52.893 INFO:teuthology.orchestra.run.smithi164.stdout:3935 write 0x8a20400 thru 0x8a288dd (0x84de bytes) 2024-09-18T01:03:52.894 INFO:teuthology.orchestra.run.smithi164.stdout:3936 read 0x82b7c00 thru 0x82bc203 (0x4604 bytes) 2024-09-18T01:03:52.894 INFO:teuthology.orchestra.run.smithi164.stdout:3937 read 0xab45000 thru 0xab50bcc (0xbbcd bytes) 2024-09-18T01:03:52.894 INFO:teuthology.orchestra.run.smithi164.stdout:3939 read 0x18b4600 thru 0x18beeb2 (0xa8b3 bytes) 2024-09-18T01:03:52.894 INFO:teuthology.orchestra.run.smithi164.stdout:3941 read 0x29fc200 thru 0x2a05a44 (0x9845 bytes) 2024-09-18T01:03:52.895 INFO:teuthology.orchestra.run.smithi164.stdout:3942 write 0x4218c00 thru 0x422625f (0xd660 bytes) 2024-09-18T01:03:52.895 INFO:teuthology.orchestra.run.smithi164.stdout:3946 read 0x3508e00 thru 0x351118a (0x838b bytes) 2024-09-18T01:03:52.895 INFO:teuthology.orchestra.run.smithi164.stdout:3948 punch from 0x1db0400 to 0x1db8a00, (0x8600 bytes) 2024-09-18T01:03:52.896 INFO:teuthology.orchestra.run.smithi164.stdout:3950 read 0x70fc200 thru 0x70fd451 (0x1252 bytes) 2024-09-18T01:03:52.897 INFO:teuthology.orchestra.run.smithi164.stdout:3951 read 0x61600 thru 0x69188 (0x7b89 bytes) 2024-09-18T01:03:52.897 INFO:teuthology.orchestra.run.smithi164.stdout:3955 punch from 0x170fc00 to 0x1711000, (0x1400 bytes) 2024-09-18T01:03:52.897 INFO:teuthology.orchestra.run.smithi164.stdout:3957 punch from 0x2199c00 to 0x21a6e00, (0xd200 bytes) 2024-09-18T01:03:52.900 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:52.899+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:52.905 INFO:teuthology.orchestra.run.smithi164.stdout:3958 trunc from 0xb333c00 to 0x4814e00 2024-09-18T01:03:52.930 INFO:teuthology.orchestra.run.smithi164.stdout:3960 trunc from 0x4814e00 to 0x5782000 2024-09-18T01:03:52.958 INFO:teuthology.orchestra.run.smithi164.stdout:3961 write 0x8ba8a00 thru 0x8bac096 (0x3697 bytes) 2024-09-18T01:03:52.959 INFO:teuthology.orchestra.run.smithi164.stdout:3962 read 0xe08600 thru 0xe10a0a (0x840b bytes) 2024-09-18T01:03:52.960 INFO:teuthology.orchestra.run.smithi164.stdout:3963 read 0x4a53c00 thru 0x4a5f7aa (0xbbab bytes) 2024-09-18T01:03:52.977 INFO:teuthology.orchestra.run.smithi164.stdout:3966 write 0xbb3ec00 thru 0xbb4697b (0x7d7c bytes) 2024-09-18T01:03:52.978 INFO:teuthology.orchestra.run.smithi164.stdout:3971 read 0x6dfe00 thru 0x6e7ca2 (0x7ea3 bytes) 2024-09-18T01:03:52.979 INFO:teuthology.orchestra.run.smithi164.stdout:3972 write 0x246ee00 thru 0x2478414 (0x9615 bytes) 2024-09-18T01:03:52.979 INFO:teuthology.orchestra.run.smithi164.stdout:3974 read 0x331c200 thru 0x33206bb (0x44bc bytes) 2024-09-18T01:03:52.979 INFO:teuthology.orchestra.run.smithi164.stdout:3975 read 0x9e17200 thru 0x9e17414 (0x215 bytes) 2024-09-18T01:03:52.980 INFO:teuthology.orchestra.run.smithi164.stdout:3978 trunc from 0xbb46a00 to 0xb885400 2024-09-18T01:03:52.986 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:52.985+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:52.998 INFO:teuthology.orchestra.run.smithi164.stdout:3979 write 0x87f4000 thru 0x8801222 (0xd223 bytes) 2024-09-18T01:03:53.000 INFO:teuthology.orchestra.run.smithi164.stdout:3981 read 0x761f000 thru 0x7623670 (0x4671 bytes) 2024-09-18T01:03:53.000 INFO:teuthology.orchestra.run.smithi164.stdout:3985 read 0xaf1000 thru 0xaf6720 (0x5721 bytes) 2024-09-18T01:03:53.000 INFO:teuthology.orchestra.run.smithi164.stdout:3986 write 0xb0d8c00 thru 0xb0e7bdb (0xefdc bytes) 2024-09-18T01:03:53.000 INFO:teuthology.orchestra.run.smithi164.stdout:3987 write 0x22baa00 thru 0x22bfb28 (0x5129 bytes) 2024-09-18T01:03:53.001 INFO:teuthology.orchestra.run.smithi164.stdout:3988 trunc from 0xb885400 to 0xbaa1000 2024-09-18T01:03:53.015 INFO:teuthology.orchestra.run.smithi164.stdout:3989 write 0xa7bb600 thru 0xa7bd3ad (0x1dae bytes) 2024-09-18T01:03:53.016 INFO:teuthology.orchestra.run.smithi164.stdout:3990 read 0x4aca200 thru 0x4ad8f8a (0xed8b bytes) 2024-09-18T01:03:53.016 INFO:teuthology.orchestra.run.smithi164.stdout:3991 write 0x3395200 thru 0x33990c5 (0x3ec6 bytes) 2024-09-18T01:03:53.017 INFO:teuthology.orchestra.run.smithi164.stdout:3993 write 0x89f9200 thru 0x8a04f12 (0xbd13 bytes) 2024-09-18T01:03:53.020 INFO:teuthology.orchestra.run.smithi164.stdout:3995 trunc from 0xbaa1000 to 0xe60d400 2024-09-18T01:03:53.024 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:53.025 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:53.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:53.035 INFO:teuthology.orchestra.run.smithi164.stdout:3998 read 0x29c5200 thru 0x29d1426 (0xc227 bytes) 2024-09-18T01:03:53.036 INFO:teuthology.orchestra.run.smithi164.stdout:3999 read 0x8ea2200 thru 0x8eaeb20 (0xc921 bytes) 2024-09-18T01:03:53.036 INFO:teuthology.orchestra.run.smithi164.stdout:4000 read 0x9d0fa00 thru 0x9d1a548 (0xab49 bytes) 2024-09-18T01:03:53.036 INFO:teuthology.orchestra.run.smithi164.stdout:4001 write 0xa356800 thru 0xa35fc15 (0x9416 bytes) 2024-09-18T01:03:53.037 INFO:teuthology.orchestra.run.smithi164.stdout:4003 write 0xaa89000 thru 0xaa96e17 (0xde18 bytes) 2024-09-18T01:03:53.037 INFO:teuthology.orchestra.run.smithi164.stdout:4004 punch from 0xd54d200 to 0xd556000, (0x8e00 bytes) 2024-09-18T01:03:53.038 INFO:teuthology.orchestra.run.smithi164.stdout:4005 write 0xbdc4000 thru 0xbdc537e (0x137f bytes) 2024-09-18T01:03:53.038 INFO:teuthology.orchestra.run.smithi164.stdout:4008 read 0x5041a00 thru 0x5049d23 (0x8324 bytes) 2024-09-18T01:03:53.038 INFO:teuthology.orchestra.run.smithi164.stdout:4010 write 0xd696400 thru 0xd6a308a (0xcc8b bytes) 2024-09-18T01:03:53.038 INFO:teuthology.orchestra.run.smithi164.stdout:4011 write 0x468d000 thru 0x469ca2b (0xfa2c bytes) 2024-09-18T01:03:53.038 INFO:teuthology.orchestra.run.smithi164.stdout:4013 write 0xbff000 thru 0xc0b8d0 (0xc8d1 bytes) 2024-09-18T01:03:53.039 INFO:teuthology.orchestra.run.smithi164.stdout:4014 read 0xab25200 thru 0xab2f6eb (0xa4ec bytes) 2024-09-18T01:03:53.039 INFO:teuthology.orchestra.run.smithi164.stdout:4015 read 0x4fc3800 thru 0x4fc658e (0x2d8f bytes) 2024-09-18T01:03:53.040 INFO:teuthology.orchestra.run.smithi164.stdout:4016 write 0x8683c00 thru 0x869260e (0xea0f bytes) 2024-09-18T01:03:53.040 INFO:teuthology.orchestra.run.smithi164.stdout:4018 write 0xba7d600 thru 0xba83ba5 (0x65a6 bytes) 2024-09-18T01:03:53.040 INFO:teuthology.orchestra.run.smithi164.stdout:4021 read 0x8466400 thru 0x846d342 (0x6f43 bytes) 2024-09-18T01:03:53.040 INFO:teuthology.orchestra.run.smithi164.stdout:4023 read 0xafc1600 thru 0xafce401 (0xce02 bytes) 2024-09-18T01:03:53.043 INFO:teuthology.orchestra.run.smithi164.stdout:4025 read 0x46f3e00 thru 0x46f6de4 (0x2fe5 bytes) 2024-09-18T01:03:53.043 INFO:teuthology.orchestra.run.smithi164.stdout:4027 punch from 0x4ec6e00 to 0x4ece600, (0x7800 bytes) 2024-09-18T01:03:53.053 INFO:teuthology.orchestra.run.smithi164.stdout:4028 trunc from 0xe60d400 to 0x6878600 2024-09-18T01:03:53.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:53.078 INFO:teuthology.orchestra.run.smithi164.stdout:4029 read 0x3dcbc00 thru 0x3dd4e7b (0x927c bytes) 2024-09-18T01:03:53.086 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:53.085+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:53.088 INFO:teuthology.orchestra.run.smithi164.stdout:4030 trunc from 0x6878600 to 0xdd8ee00 2024-09-18T01:03:53.105 INFO:teuthology.orchestra.run.smithi164.stdout:4031 punch from 0x4054800 to 0x405d600, (0x8e00 bytes) 2024-09-18T01:03:53.105 INFO:teuthology.orchestra.run.smithi164.stdout:4033 write 0x8244e00 thru 0x82496af (0x48b0 bytes) 2024-09-18T01:03:53.105 INFO:teuthology.orchestra.run.smithi164.stdout:4035 read 0xe91400 thru 0xe98293 (0x6e94 bytes) 2024-09-18T01:03:53.120 INFO:teuthology.orchestra.run.smithi164.stdout:4038 trunc from 0xdd8ee00 to 0x176a400 2024-09-18T01:03:53.157 INFO:teuthology.orchestra.run.smithi164.stdout:4039 trunc from 0x176a400 to 0xa6a2c00 2024-09-18T01:03:53.166 INFO:teuthology.orchestra.run.smithi164.stdout:4040 read 0x1e8d800 thru 0x1e989e1 (0xb1e2 bytes) 2024-09-18T01:03:53.173 INFO:teuthology.orchestra.run.smithi164.stdout:4041 write 0x62d3e00 thru 0x62d59cc (0x1bcd bytes) 2024-09-18T01:03:53.188 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:53.186+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:53.194 INFO:teuthology.orchestra.run.smithi164.stdout:4043 write 0xd4f9a00 thru 0xd505164 (0xb765 bytes) 2024-09-18T01:03:53.194 INFO:teuthology.orchestra.run.smithi164.stdout:4044 write 0xb3ee00 thru 0xb4e08c (0xf28d bytes) 2024-09-18T01:03:53.202 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:53.202 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:53.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:53.202 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.465733563000001, 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:53.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:53.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:53.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.465713604999998, 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:53.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.465694663999997, 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:53.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:53.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.465673154999998, 2024-09-18T01:03:53.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:53.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:53.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:53.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.465647427, 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:53.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:53.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456915567999999, 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:53.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456892854000003, 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:53.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456873723999998, 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:53.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:53.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:53.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456854878000001, 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:53.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:53.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456836000000003, 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:53.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.265 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456814135000002, 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:53.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:53.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456790499999997, 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:53.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:53.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:53.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456756849999998, 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:53.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456728994999999, 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:53.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456709171999996, 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456688737999997, 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:53.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:53.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:53.303 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456666658000003, 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:53.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456644335999997, 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:53.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:53.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456621939999998, 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:53.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.456601521000003, 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:53.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:53.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:53.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.328 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:53.331 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:53.331 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:53.332 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:53.332 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:53.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:53.333 INFO:teuthology.orchestra.run.smithi164.stdout:4046 trunc from 0xd505200 to 0x1070400 2024-09-18T01:03:53.333 INFO:teuthology.orchestra.run.smithi164.stdout:4047 write 0x86c7a00 thru 0x86cd4e9 (0x5aea bytes) 2024-09-18T01:03:53.333 INFO:teuthology.orchestra.run.smithi164.stdout:4049 write 0x27fd800 thru 0x2802bd9 (0x53da bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4050 read 0x57ef400 thru 0x57f9ed4 (0xaad5 bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4051 read 0x5d83200 thru 0x5d91dde (0xebdf bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4053 read 0x2d1c200 thru 0x2d1c24e (0x4f bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4054 write 0xc0d1800 thru 0xc0d3ef3 (0x26f4 bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4058 write 0xaf6e200 thru 0xaf7a4ea (0xc2eb bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4060 write 0x5ae0200 thru 0x5aef6e8 (0xf4e9 bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4062 trunc from 0xc0d4000 to 0xdde8600 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4063 write 0x5d92800 thru 0x5d9898c (0x618d bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4066 read 0x800b800 thru 0x800e986 (0x3187 bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4067 read 0xa75fa00 thru 0xa76f64e (0xfc4f bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4068 read 0x3422600 thru 0x3429abf (0x74c0 bytes) 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi164.stdout:4070 trunc from 0xdde8600 to 0x1ecf600 2024-09-18T01:03:53.334 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:53.335 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:53.335 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:03:53.336 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:53.337 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:53.328+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:53.351 INFO:teuthology.orchestra.run.smithi164.stdout:4071 read 0x1bad400 thru 0x1bb4265 (0x6e66 bytes) 2024-09-18T01:03:53.352 INFO:teuthology.orchestra.run.smithi164.stdout:4072 read 0x6e1600 thru 0x6ee42a (0xce2b bytes) 2024-09-18T01:03:53.353 INFO:teuthology.orchestra.run.smithi164.stdout:4073 read 0xd74c00 thru 0xd7a280 (0x5681 bytes) 2024-09-18T01:03:53.353 INFO:teuthology.orchestra.run.smithi164.stdout:4077 read 0xe72e00 thru 0xe7b61c (0x881d bytes) 2024-09-18T01:03:53.353 INFO:teuthology.orchestra.run.smithi164.stdout:4079 read 0xaae400 thru 0xab48ee (0x64ef bytes) 2024-09-18T01:03:53.369 INFO:teuthology.orchestra.run.smithi164.stdout:4080 write 0x43e1c00 thru 0x43eedac (0xd1ad bytes) 2024-09-18T01:03:53.372 INFO:teuthology.orchestra.run.smithi164.stdout:4081 trunc from 0x43eee00 to 0x70c2c00 2024-09-18T01:03:53.386 INFO:teuthology.orchestra.run.smithi164.stdout:4082 read 0x5afb200 thru 0x5b0859f (0xd3a0 bytes) 2024-09-18T01:03:53.388 INFO:teuthology.orchestra.run.smithi164.stdout:4085 read 0x2265400 thru 0x22720d7 (0xccd8 bytes) 2024-09-18T01:03:53.388 INFO:teuthology.orchestra.run.smithi164.stdout:4086 write 0x206a00 thru 0x213cbf (0xd2c0 bytes) 2024-09-18T01:03:53.402 INFO:teuthology.orchestra.run.smithi164.stdout:4089 write 0x8975c00 thru 0x897e5e8 (0x89e9 bytes) 2024-09-18T01:03:53.403 INFO:teuthology.orchestra.run.smithi164.stdout:4090 read 0x76ef000 thru 0x76f4893 (0x5894 bytes) 2024-09-18T01:03:53.409 INFO:teuthology.orchestra.run.smithi164.stdout:4091 trunc from 0x897e600 to 0x326cc00 2024-09-18T01:03:53.433 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:53.432+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:53.434 INFO:teuthology.orchestra.run.smithi164.stdout:4093 read 0x16cb200 thru 0x16d6a4c (0xb84d bytes) 2024-09-18T01:03:53.445 INFO:teuthology.orchestra.run.smithi164.stdout:4094 write 0x404c400 thru 0x4057bac (0xb7ad bytes) 2024-09-18T01:03:53.452 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:53.453 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:53.462 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:53.470 INFO:teuthology.orchestra.run.smithi164.stdout:4095 write 0x80a1000 thru 0x80a4e23 (0x3e24 bytes) 2024-09-18T01:03:53.471 INFO:teuthology.orchestra.run.smithi164.stdout:4096 read 0x17fcc00 thru 0x17fd8dd (0xcde bytes) 2024-09-18T01:03:53.472 INFO:teuthology.orchestra.run.smithi164.stdout:4099 write 0x7e68000 thru 0x7e6a266 (0x2267 bytes) 2024-09-18T01:03:53.480 INFO:teuthology.orchestra.run.smithi164.stdout:4101 trunc from 0x80a5000 to 0xead5800 2024-09-18T01:03:53.492 INFO:teuthology.orchestra.run.smithi164.stdout:4105 write 0x137fa00 thru 0x138ce45 (0xd446 bytes) 2024-09-18T01:03:53.494 INFO:teuthology.orchestra.run.smithi164.stdout:4106 punch from 0x3d23800 to 0x3d25e00, (0x2600 bytes) 2024-09-18T01:03:53.495 INFO:teuthology.orchestra.run.smithi164.stdout:4107 read 0x1d9e200 thru 0x1da942e (0xb22f bytes) 2024-09-18T01:03:53.495 INFO:teuthology.orchestra.run.smithi164.stdout:4108 write 0x2981400 thru 0x29869ce (0x55cf bytes) 2024-09-18T01:03:53.496 INFO:teuthology.orchestra.run.smithi164.stdout:4109 trunc from 0xead5800 to 0xe79d800 2024-09-18T01:03:53.507 INFO:teuthology.orchestra.run.smithi164.stdout:4111 trunc from 0xe79d800 to 0xe52b200 2024-09-18T01:03:53.518 INFO:teuthology.orchestra.run.smithi164.stdout:4112 write 0x249200 thru 0x2535e4 (0xa3e5 bytes) 2024-09-18T01:03:53.520 INFO:teuthology.orchestra.run.smithi164.stdout:4113 read 0x5e77400 thru 0x5e7b695 (0x4296 bytes) 2024-09-18T01:03:53.520 INFO:teuthology.orchestra.run.smithi164.stdout:4117 read 0x8f21a00 thru 0x8f2d6d6 (0xbcd7 bytes) 2024-09-18T01:03:53.520 INFO:teuthology.orchestra.run.smithi164.stdout:4118 punch from 0xfb7a00 to 0xfc7400, (0xfa00 bytes) 2024-09-18T01:03:53.521 INFO:teuthology.orchestra.run.smithi164.stdout:4119 punch from 0x242fa00 to 0x2436800, (0x6e00 bytes) 2024-09-18T01:03:53.521 INFO:teuthology.orchestra.run.smithi164.stdout:4121 read 0xc52ac00 thru 0xc53188c (0x6c8d bytes) 2024-09-18T01:03:53.521 INFO:teuthology.orchestra.run.smithi164.stdout:4123 read 0x7d70600 thru 0x7d73fb9 (0x39ba bytes) 2024-09-18T01:03:53.521 INFO:teuthology.orchestra.run.smithi164.stdout:4124 punch from 0x920a800 to 0x920d800, (0x3000 bytes) 2024-09-18T01:03:53.521 INFO:teuthology.orchestra.run.smithi164.stdout:4127 punch from 0xcfc600 to 0xd0a800, (0xe200 bytes) 2024-09-18T01:03:53.522 INFO:teuthology.orchestra.run.smithi164.stdout:4129 punch from 0xe34f600 to 0xe350200, (0xc00 bytes) 2024-09-18T01:03:53.522 INFO:teuthology.orchestra.run.smithi164.stdout:4130 read 0x7bc6000 thru 0x7bcd405 (0x7406 bytes) 2024-09-18T01:03:53.525 INFO:teuthology.orchestra.run.smithi164.stdout:4131 write 0xb474400 thru 0xb47cd75 (0x8976 bytes) 2024-09-18T01:03:53.525 INFO:teuthology.orchestra.run.smithi164.stdout:4132 read 0x37c200 thru 0x382cc6 (0x6ac7 bytes) 2024-09-18T01:03:53.526 INFO:teuthology.orchestra.run.smithi164.stdout:4133 read 0x3374200 thru 0x337f037 (0xae38 bytes) 2024-09-18T01:03:53.526 INFO:teuthology.orchestra.run.smithi164.stdout:4134 read 0x98e5600 thru 0x98e8dca (0x37cb bytes) 2024-09-18T01:03:53.538 INFO:teuthology.orchestra.run.smithi164.stdout:4137 write 0xe734000 thru 0xe7365ea (0x25eb bytes) 2024-09-18T01:03:53.540 INFO:teuthology.orchestra.run.smithi164.stdout:4138 read 0x9220400 thru 0x92259d8 (0x55d9 bytes) 2024-09-18T01:03:53.540 INFO:teuthology.orchestra.run.smithi164.stdout:4140 read 0xc389c00 thru 0xc395bd3 (0xbfd4 bytes) 2024-09-18T01:03:53.540 INFO:teuthology.orchestra.run.smithi164.stdout:4143 write 0xb79ec00 thru 0xb7ab226 (0xc627 bytes) 2024-09-18T01:03:53.546 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:53.545+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:53.549 INFO:teuthology.orchestra.run.smithi164.stdout:4146 trunc from 0xe736600 to 0x69bd600 2024-09-18T01:03:53.551 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:53.552 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:53.552 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T01:03:53.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:03:53.573 INFO:teuthology.orchestra.run.smithi164.stdout:4147 read 0x19c1800 thru 0x19c3f9d (0x279e bytes) 2024-09-18T01:03:53.574 INFO:teuthology.orchestra.run.smithi164.stdout:4148 read 0x3fb5000 thru 0x3fbaca3 (0x5ca4 bytes) 2024-09-18T01:03:53.574 INFO:teuthology.orchestra.run.smithi164.stdout:4149 read 0x7a5a00 thru 0x7b4f43 (0xf544 bytes) 2024-09-18T01:03:53.575 INFO:teuthology.orchestra.run.smithi164.stdout:4150 punch from 0x2318400 to 0x2326c00, (0xe800 bytes) 2024-09-18T01:03:53.593 INFO:teuthology.orchestra.run.smithi164.stdout:4151 write 0x7bb8200 thru 0x7bbf953 (0x7754 bytes) 2024-09-18T01:03:53.596 INFO:teuthology.orchestra.run.smithi164.stdout:4152 trunc from 0x7bbfa00 to 0x5678400 2024-09-18T01:03:53.620 INFO:teuthology.orchestra.run.smithi164.stdout:4153 read 0x3c40600 thru 0x3c4533e (0x4d3f bytes) 2024-09-18T01:03:53.620 INFO:teuthology.orchestra.run.smithi164.stdout:4154 write 0x967800 thru 0x96d832 (0x6033 bytes) 2024-09-18T01:03:53.621 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:53.621 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:53.621 INFO:teuthology.orchestra.run.smithi164.stdout:4155 punch from 0xef6800 to 0xefe800, (0x8000 bytes) 2024-09-18T01:03:53.622 INFO:teuthology.orchestra.run.smithi164.stdout:4156 read 0x3c02400 thru 0x3c070af (0x4cb0 bytes) 2024-09-18T01:03:53.622 INFO:teuthology.orchestra.run.smithi164.stdout:4157 read 0xe96600 thru 0xea1d95 (0xb796 bytes) 2024-09-18T01:03:53.628 INFO:teuthology.orchestra.run.smithi164.stdout:4158 trunc from 0x5678400 to 0x141a600 2024-09-18T01:03:53.630 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:53.635 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:53.634+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:53.661 INFO:teuthology.orchestra.run.smithi164.stdout:4160 trunc from 0x141a600 to 0x8d49c00 2024-09-18T01:03:53.673 INFO:teuthology.orchestra.run.smithi164.stdout:4163 punch from 0x24e3000 to 0x24eaa00, (0x7a00 bytes) 2024-09-18T01:03:53.682 INFO:teuthology.orchestra.run.smithi164.stdout:4165 trunc from 0x8d49c00 to 0x6742400 2024-09-18T01:03:53.695 INFO:teuthology.orchestra.run.smithi164.stdout:4167 trunc from 0x6742400 to 0x73a1000 2024-09-18T01:03:53.734 INFO:teuthology.orchestra.run.smithi164.stdout:4169 write 0xa012e00 thru 0xa01320d (0x40e bytes) 2024-09-18T01:03:53.734 INFO:teuthology.orchestra.run.smithi164.stdout:4170 read 0x1ff1200 thru 0x1ff7fe1 (0x6de2 bytes) 2024-09-18T01:03:53.734 INFO:teuthology.orchestra.run.smithi164.stdout:4171 write 0x5dc8e00 thru 0x5dcf499 (0x669a bytes) 2024-09-18T01:03:53.735 INFO:teuthology.orchestra.run.smithi164.stdout:4175 punch from 0x5661000 to 0x5666400, (0x5400 bytes) 2024-09-18T01:03:53.735 INFO:teuthology.orchestra.run.smithi164.stdout:4177 write 0x3fd2a00 thru 0x3fd4442 (0x1a43 bytes) 2024-09-18T01:03:53.735 INFO:teuthology.orchestra.run.smithi164.stdout:4179 read 0x8565800 thru 0x856c06d (0x686e bytes) 2024-09-18T01:03:53.736 INFO:teuthology.orchestra.run.smithi164.stdout:4183 write 0x5c83800 thru 0x5c91d05 (0xe506 bytes) 2024-09-18T01:03:53.736 INFO:teuthology.orchestra.run.smithi164.stdout:4184 write 0x5f9c000 thru 0x5f9ffcf (0x3fd0 bytes) 2024-09-18T01:03:53.736 INFO:teuthology.orchestra.run.smithi164.stdout:4186 read 0xd46a00 thru 0xd5250b (0xbb0c bytes) 2024-09-18T01:03:53.736 INFO:teuthology.orchestra.run.smithi164.stdout:4191 punch from 0x1f7e200 to 0x1f86a00, (0x8800 bytes) 2024-09-18T01:03:53.737 INFO:teuthology.orchestra.run.smithi164.stdout:4192 write 0x87ed200 thru 0x87f7da3 (0xaba4 bytes) 2024-09-18T01:03:53.737 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:53.736+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:53.738 INFO:teuthology.orchestra.run.smithi164.stdout:4193 trunc from 0xa013400 to 0xbf20c00 2024-09-18T01:03:53.765 INFO:teuthology.orchestra.run.smithi164.stdout:4194 read 0x9218400 thru 0x9226a26 (0xe627 bytes) 2024-09-18T01:03:53.768 INFO:teuthology.orchestra.run.smithi164.stdout:4195 trunc from 0xbf20c00 to 0x9722400 2024-09-18T01:03:53.782 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:53.782 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:53.787 INFO:teuthology.orchestra.run.smithi164.stdout:4196 read 0x8e9b800 thru 0x8e9ef3f (0x3740 bytes) 2024-09-18T01:03:53.787 INFO:teuthology.orchestra.run.smithi164.stdout:4198 write 0x54a1000 thru 0x54a8ea6 (0x7ea7 bytes) 2024-09-18T01:03:53.788 INFO:teuthology.orchestra.run.smithi164.stdout:4199 read 0x49db600 thru 0x49df4ea (0x3eeb bytes) 2024-09-18T01:03:53.792 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:53.794 INFO:teuthology.orchestra.run.smithi164.stdout:4201 trunc from 0x9722400 to 0x47cd800 2024-09-18T01:03:53.817 INFO:teuthology.orchestra.run.smithi164.stdout:4207 trunc from 0x47cd800 to 0x2e9ec00 2024-09-18T01:03:53.838 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:53.837+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:53.860 INFO:teuthology.orchestra.run.smithi164.stdout:4208 write 0x70b3000 thru 0x70b5a7c (0x2a7d bytes) 2024-09-18T01:03:53.870 INFO:teuthology.orchestra.run.smithi164.stdout:4209 trunc from 0x70b5c00 to 0xe466400 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 81, 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-18T01:02:13.493841+0000", 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-18T01:03:53.306597+0000", 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-18T01:02:24.290620+0000", 2024-09-18T01:03:53.878 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-18T01:02:48.274416+0000", 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-18T01:03:53.879 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-18T01:03:53.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:24.388064+0000", 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:53.881 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:53.882 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:53.883 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:53.884 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:53.885 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:53.886 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:30.676199+0000", 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:53.888 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:53.889 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-18T01:03:53.890 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:53.891 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:53.892 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2699999809265137, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2699999809265137, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:53.893 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:40.390732+0000", 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:03:53.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:03:53.895 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.896 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "80", 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 28, 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 80, 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:03:53.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:03:53.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:03:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.7000000476837158, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.7000000476837158, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:03:53.900 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4563fb16-8745-4e75-b654-1e96403d91b9", 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.8100128173828125, 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:53.901 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/1626166637", 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/1626166637", 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/1626166637", 2024-09-18T01:03:53.906 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/1626166637", 2024-09-18T01:03:53.907 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:53.907 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:53.907 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:53.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.907 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:53.908 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "634965cb-d444-4721-b34a-afbc49105cab", 2024-09-18T01:03:53.908 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:53.908 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:53.908 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:53.908 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:53.908 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.911 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/2210970259", 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/2210970259", 2024-09-18T01:03:53.913 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/2210970259", 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/2210970259", 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "83443eab-2fa4-4c7e-a459-edb1bb0ed8d0", 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.6881561279296875, 2024-09-18T01:03:53.914 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-18T01:03:53.916 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:53.918 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/617133675", 2024-09-18T01:03:53.919 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/617133675", 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/617133675", 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/617133675", 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "5797982a-71c1-4d6a-9604-00fbc6bd16d9", 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:53.920 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6808", 2024-09-18T01:03:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6809", 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.922 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6810", 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6811", 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6814", 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6815", 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6812", 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6813", 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6809/151031464", 2024-09-18T01:03:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6811/151031464", 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6815/151031464", 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6813/151031464", 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f79e3329-5481-4f33-8d28-6386af186492", 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:03:53.926 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 74, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6816", 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6817", 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6818", 2024-09-18T01:03:53.928 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6819", 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.929 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6822", 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6823", 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6820", 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6821", 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6817/1499036965", 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6819/1499036965", 2024-09-18T01:03:53.931 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6823/1499036965", 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6821/1499036965", 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "8c61d2ec-b111-4783-a016-336ed33a0694", 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-18T01:03:53.932 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 20, 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6800", 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6801", 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6802", 2024-09-18T01:03:53.934 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6803", 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.935 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6806", 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6807", 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:03:53.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6804", 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6805", 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6801/2310498524", 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6803/2310498524", 2024-09-18T01:03:53.937 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6807/2310498524", 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6805/2310498524", 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:53.938 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.877700+0000", 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:53.939 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.360930+0000", 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.403205+0000", 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.952123+0000", 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:53.941 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:22.099545+0000", 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-18T01:03:53.942 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.694132+0000", 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-18T01:03:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-18T01:03:53.944 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-18T01:03:53.945 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:03:53.945 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:03:53.945 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-18T01:03:53.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:53.945 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:53.947 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:53.947 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:53.948 INFO:tasks.thrashosds.thrasher:j is {'epoch': 81, 'fsid': '62b46b83-8ef2-466f-b28b-2d97f265422e', 'created': '2024-09-18T01:02:13.493841+0000', 'modified': '2024-09-18T01:03:53.306597+0000', 'last_up_change': '2024-09-18T01:02:24.290620+0000', 'last_in_change': '2024-09-18T01:02:48.274416+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:02:24.388064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:02:30.676199+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:02:40.390732+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 80, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4563fb16-8745-4e75-b654-1e96403d91b9', 'up': 1, 'in': 1, 'weight': 0.8100128173828125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 1626166637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 1626166637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 1626166637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 1626166637}]}, 'public_addr': '172.21.15.31:6817/1626166637', 'cluster_addr': '172.21.15.31:6819/1626166637', 'heartbeat_back_addr': '172.21.15.31:6823/1626166637', 'heartbeat_front_addr': '172.21.15.31:6821/1626166637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '634965cb-d444-4721-b34a-afbc49105cab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 2210970259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 2210970259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 2210970259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 2210970259}]}, 'public_addr': '172.21.15.31:6801/2210970259', 'cluster_addr': '172.21.15.31:6803/2210970259', 'heartbeat_back_addr': '172.21.15.31:6807/2210970259', 'heartbeat_front_addr': '172.21.15.31:6805/2210970259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'up': 1, 'in': 1, 'weight': 0.6881561279296875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 617133675}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 617133675}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 617133675}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 617133675}]}, 'public_addr': '172.21.15.31:6809/617133675', 'cluster_addr': '172.21.15.31:6811/617133675', 'heartbeat_back_addr': '172.21.15.31:6815/617133675', 'heartbeat_front_addr': '172.21.15.31:6813/617133675', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6808', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6809', 'nonce': 151031464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6810', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6811', 'nonce': 151031464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6814', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6815', 'nonce': 151031464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6812', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6813', 'nonce': 151031464}]}, 'public_addr': '172.21.15.98:6809/151031464', 'cluster_addr': '172.21.15.98:6811/151031464', 'heartbeat_back_addr': '172.21.15.98:6815/151031464', 'heartbeat_front_addr': '172.21.15.98:6813/151031464', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f79e3329-5481-4f33-8d28-6386af186492', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6816', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6817', 'nonce': 1499036965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6818', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6819', 'nonce': 1499036965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6822', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6823', 'nonce': 1499036965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6820', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6821', 'nonce': 1499036965}]}, 'public_addr': '172.21.15.98:6817/1499036965', 'cluster_addr': '172.21.15.98:6819/1499036965', 'heartbeat_back_addr': '172.21.15.98:6823/1499036965', 'heartbeat_front_addr': '172.21.15.98:6821/1499036965', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8c61d2ec-b111-4783-a016-336ed33a0694', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6800', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6801', 'nonce': 2310498524}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6802', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6803', 'nonce': 2310498524}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6806', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6807', 'nonce': 2310498524}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6804', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6805', 'nonce': 2310498524}]}, 'public_addr': '172.21.15.98:6801/2310498524', 'cluster_addr': '172.21.15.98:6803/2310498524', 'heartbeat_back_addr': '172.21.15.98:6807/2310498524', 'heartbeat_front_addr': '172.21.15.98:6805/2310498524', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.877700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.360930+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.403205+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.952123+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:22.099545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.694132+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:03:53.948 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T01:03:53.949 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:53.945+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:53.949 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:53.949 INFO:teuthology.orchestra.run.smithi164.stdout:4210 trunc from 0xe466400 to 0x3b4a800 2024-09-18T01:03:53.949 INFO:teuthology.orchestra.run.smithi164.stdout:4212 write 0xbc9c200 thru 0xbca2137 (0x5f38 bytes) 2024-09-18T01:03:53.949 INFO:teuthology.orchestra.run.smithi164.stdout:4215 punch from 0x8245c00 to 0x824c800, (0x6c00 bytes) 2024-09-18T01:03:53.950 INFO:teuthology.orchestra.run.smithi164.stdout:4218 write 0x112800 thru 0x1135dc (0xddd bytes) 2024-09-18T01:03:53.950 INFO:teuthology.orchestra.run.smithi164.stdout:4221 write 0x4c9e600 thru 0x4c9fd32 (0x1733 bytes) 2024-09-18T01:03:53.950 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xede1200 2024-09-18T01:03:53.950 INFO:teuthology.orchestra.run.smithi164.stdout:4222 trunc from 0xbca2200 to 0xede1200 2024-09-18T01:03:53.953 INFO:teuthology.orchestra.run.smithi164.stdout:4227 write 0xbb4ac00 thru 0xbb55b32 (0xaf33 bytes) 2024-09-18T01:03:53.953 INFO:teuthology.orchestra.run.smithi164.stdout:4230 read 0x688f400 thru 0x689ed71 (0xf972 bytes) 2024-09-18T01:03:53.953 INFO:teuthology.orchestra.run.smithi164.stdout:4233 read 0x46d7800 thru 0x46dc765 (0x4f66 bytes) 2024-09-18T01:03:53.953 INFO:teuthology.orchestra.run.smithi164.stdout:4234 read 0x6c9e400 thru 0x6caaa11 (0xc612 bytes) 2024-09-18T01:03:53.955 INFO:teuthology.orchestra.run.smithi164.stdout:4237 trunc from 0xede1200 to 0xdad9200 2024-09-18T01:03:53.976 INFO:teuthology.orchestra.run.smithi164.stdout:4239 write 0xc15d600 thru 0xc163df8 (0x67f9 bytes) 2024-09-18T01:03:53.976 INFO:teuthology.orchestra.run.smithi164.stdout:4240 write 0x7e76e00 thru 0x7e7a189 (0x338a bytes) 2024-09-18T01:03:53.977 INFO:teuthology.orchestra.run.smithi164.stdout:4241 read 0x3a4b200 thru 0x3a5596a (0xa76b bytes) 2024-09-18T01:03:53.977 INFO:teuthology.orchestra.run.smithi164.stdout:4244 punch from 0x74d8200 to 0x74e2800, (0xa600 bytes) 2024-09-18T01:03:53.977 INFO:teuthology.orchestra.run.smithi164.stdout:4245 write 0xb721600 thru 0xb729757 (0x8158 bytes) 2024-09-18T01:03:53.977 INFO:teuthology.orchestra.run.smithi164.stdout:4247 read 0x8281c00 thru 0x828d356 (0xb757 bytes) 2024-09-18T01:03:53.977 INFO:teuthology.orchestra.run.smithi164.stdout:4248 punch from 0x6a3e200 to 0x6a41600, (0x3400 bytes) 2024-09-18T01:03:53.978 INFO:teuthology.orchestra.run.smithi164.stdout:4249 read 0x291fa00 thru 0x292e760 (0xed61 bytes) 2024-09-18T01:03:53.993 INFO:teuthology.orchestra.run.smithi164.stdout:4250 trunc from 0xdad9200 to 0x817c00 2024-09-18T01:03:54.026 INFO:teuthology.orchestra.run.smithi164.stdout:4251 trunc from 0x817c00 to 0x4755000 2024-09-18T01:03:54.034 INFO:teuthology.orchestra.run.smithi164.stdout:4254 write 0x1909400 thru 0x191303f (0x9c40 bytes) 2024-09-18T01:03:54.035 INFO:teuthology.orchestra.run.smithi164.stdout:4255 write 0x34c5400 thru 0x34d50d3 (0xfcd4 bytes) 2024-09-18T01:03:54.035 INFO:teuthology.orchestra.run.smithi164.stdout:4256 read 0x3b5ee00 thru 0x3b60590 (0x1791 bytes) 2024-09-18T01:03:54.046 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:54.045+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:54.057 INFO:teuthology.orchestra.run.smithi164.stdout:4257 write 0xbe86200 thru 0xbe8c2ef (0x60f0 bytes) 2024-09-18T01:03:54.058 INFO:teuthology.orchestra.run.smithi164.stdout:4258 write 0x85fbc00 thru 0x8606cc6 (0xb0c7 bytes) 2024-09-18T01:03:54.058 INFO:teuthology.orchestra.run.smithi164.stdout:4262 read 0x5964e00 thru 0x596d088 (0x8289 bytes) 2024-09-18T01:03:54.058 INFO:teuthology.orchestra.run.smithi164.stdout:4264 punch from 0x3392000 to 0x3397800, (0x5800 bytes) 2024-09-18T01:03:54.059 INFO:teuthology.orchestra.run.smithi164.stdout:4267 write 0x76ba000 thru 0x76c7aca (0xdacb bytes) 2024-09-18T01:03:54.059 INFO:teuthology.orchestra.run.smithi164.stdout:4269 write 0x6246200 thru 0x6252020 (0xbe21 bytes) 2024-09-18T01:03:54.060 INFO:teuthology.orchestra.run.smithi164.stdout:4270 write 0x8aa2e00 thru 0x8aa63c1 (0x35c2 bytes) 2024-09-18T01:03:54.060 INFO:teuthology.orchestra.run.smithi164.stdout:4271 punch from 0x35ca00 to 0x35d400, (0xa00 bytes) 2024-09-18T01:03:54.066 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:54.066 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:54.070 INFO:teuthology.orchestra.run.smithi164.stdout:4273 trunc from 0xbe8c400 to 0x2c2bc00 2024-09-18T01:03:54.076 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:54.098 INFO:teuthology.orchestra.run.smithi164.stdout:4276 punch from 0x4b7e00 to 0x4c1c00, (0x9e00 bytes) 2024-09-18T01:03:54.101 INFO:teuthology.orchestra.run.smithi164.stdout:4277 read 0x2965c00 thru 0x297283b (0xcc3c bytes) 2024-09-18T01:03:54.102 INFO:teuthology.orchestra.run.smithi164.stdout:4278 clone 27 order 23 su 524288 sc 10 2024-09-18T01:03:54.160 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:54.159+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:54.219 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:54.219 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:54.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:54.268 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:54.267+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:54.350 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:54.349+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:54.374 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:54.374 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:54.384 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:54.391 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone26 from 0x2c2bc00 (overlap 0x817c00) to 0x33ca00 2024-09-18T01:03:54.469 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:54.468+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:54.530 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:54.531 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:54.539 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:54.580 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:54.579+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:54.653 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:54.652+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:54.686 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:54.686 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:54.688 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-18T01:03:54.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:54.754 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:54.753+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:54.815 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:54.815 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:54.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:54.869 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:54.868+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:54.944 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:54.944 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:54.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:54.995 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:54.994+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:55.057 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:55.056+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:55.105 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:55.105 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:55.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:55.158 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:55.157+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:55.221 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:55.221 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:55.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:55.259 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:55.258+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:55.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:03:55.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:03:55.360 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:55.359+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:55.383 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:55.383 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:55.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:55.461 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:55.459+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:55.468 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:55.485 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:55.485 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:55.495 INFO:teuthology.orchestra.run.smithi164.stdout:4279 write 0x57b8e00 thru 0x57c8010 (0xf211 bytes) 2024-09-18T01:03:55.495 INFO:teuthology.orchestra.run.smithi164.stdout:4280 read 0xdf8e00 thru 0xe04f88 (0xc189 bytes) 2024-09-18T01:03:55.495 INFO:teuthology.orchestra.run.smithi164.stdout:4281 read 0x727a00 thru 0x733a77 (0xc078 bytes) 2024-09-18T01:03:55.496 INFO:teuthology.orchestra.run.smithi164.stdout:4282 punch from 0x1cf7200 to 0x1d04800, (0xd600 bytes) 2024-09-18T01:03:55.504 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:55.505 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:55.508 INFO:teuthology.orchestra.run.smithi164.stdout:4283 trunc from 0x57c8200 to 0xd4d4a00 2024-09-18T01:03:55.518 INFO:teuthology.orchestra.run.smithi164.stdout:4284 read 0x5927400 thru 0x592ff50 (0x8b51 bytes) 2024-09-18T01:03:55.523 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:55.523 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:55.527 INFO:teuthology.orchestra.run.smithi164.stdout:4286 trunc from 0xd4d4a00 to 0x7248600 2024-09-18T01:03:55.538 INFO:teuthology.orchestra.run.smithi164.stdout:4287 write 0x2883e00 thru 0x2884240 (0x441 bytes) 2024-09-18T01:03:55.540 INFO:teuthology.orchestra.run.smithi164.stdout:4290 read 0x63cd000 thru 0x63daf87 (0xdf88 bytes) 2024-09-18T01:03:55.541 INFO:teuthology.orchestra.run.smithi164.stdout:4291 write 0x3f57a00 thru 0x3f5ef15 (0x7516 bytes) 2024-09-18T01:03:55.541 INFO:teuthology.orchestra.run.smithi164.stdout:4292 write 0x5621c00 thru 0x5622d10 (0x1111 bytes) 2024-09-18T01:03:55.541 INFO:teuthology.orchestra.run.smithi164.stdout:4295 punch from 0xe08000 to 0xe17a00, (0xfa00 bytes) 2024-09-18T01:03:55.543 INFO:teuthology.orchestra.run.smithi164.stdout:4296 punch from 0x2764c00 to 0x2773a00, (0xee00 bytes) 2024-09-18T01:03:55.543 INFO:teuthology.orchestra.run.smithi164.stdout:4299 punch from 0x5334e00 to 0x5341400, (0xc600 bytes) 2024-09-18T01:03:55.543 INFO:teuthology.orchestra.run.smithi164.stdout:4302 read 0x5ce1800 thru 0x5cef646 (0xde47 bytes) 2024-09-18T01:03:55.544 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:55.544 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:55.545 INFO:teuthology.orchestra.run.smithi164.stdout:4303 punch from 0x6cfe000 to 0x6d00800, (0x2800 bytes) 2024-09-18T01:03:55.545 INFO:teuthology.orchestra.run.smithi164.stdout:4305 punch from 0x4b6da00 to 0x4b75400, (0x7a00 bytes) 2024-09-18T01:03:55.546 INFO:teuthology.orchestra.run.smithi164.stdout:4306 punch from 0x6bcc600 to 0x6bd5e00, (0x9800 bytes) 2024-09-18T01:03:55.546 INFO:teuthology.orchestra.run.smithi164.stdout:4307 write 0x6db4200 thru 0x6dc2db4 (0xebb5 bytes) 2024-09-18T01:03:55.548 INFO:teuthology.orchestra.run.smithi164.stdout:4311 trunc from 0x7248600 to 0x8b54600 2024-09-18T01:03:55.561 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:55.560+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:55.562 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:55.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 67.282530988999994, 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:55.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:03:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:03:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:03:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:03:55.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:55.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:55.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:03:55.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 67.277905086000004, 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:03:55.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:03:55.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:55.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:03:55.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 67.277827036999994, 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:55.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:03:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:03:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:03:55.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:03:55.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:55.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.587 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.221030994000003, 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.221005130000002, 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:03:55.592 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:03:55.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.220982650000003, 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.220961613999997, 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.220934219999997, 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:03:55.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:03:55.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.220909915, 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:03:55.611 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:03:55.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217510431999997, 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:55.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217485871999997, 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:03:55.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:03:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217444354999998, 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:03:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:03:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217423853, 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217403386999997, 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:03:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:03:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217362047000002, 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:03:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:03:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:03:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:03:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217342008999999, 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:03:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:03:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:03:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:03:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:03:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:03:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:03:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:03:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:03:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217299678000003, 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:03:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:03:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:03:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:03:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:03:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:03:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:03:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:03:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:03:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217278686, 2024-09-18T01:03:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:03:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:03:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.217227715999996, 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:03:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:03:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:03:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.985087328000001, 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:03:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:03:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:03:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:03:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:03:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:03:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:03:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:55.681 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:55.682 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4312 write 0x6ae0e00 thru 0x6ae5a8f (0x4c90 bytes) 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4313 read 0x3864000 thru 0x386dee4 (0x9ee5 bytes) 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4315 punch from 0xda8000 to 0xda9200, (0x1200 bytes) 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4316 punch from 0x6068400 to 0x606e800, (0x6400 bytes) 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4318 trunc from 0x8b54600 to 0x5a66000 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4322 trunc from 0x5a66000 to 0x1c8aa00 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4323 trunc from 0x1c8aa00 to 0x6f4a00 2024-09-18T01:03:55.684 INFO:teuthology.orchestra.run.smithi164.stdout:4324 write 0xd52c000 thru 0xd53211d (0x611e bytes) 2024-09-18T01:03:55.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:55.686 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:55.681+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:55.695 INFO:teuthology.orchestra.run.smithi164.stdout:4326 trunc from 0xd532200 to 0x123d200 2024-09-18T01:03:55.738 INFO:teuthology.orchestra.run.smithi164.stdout:4331 write 0x5ddc400 thru 0x5de9829 (0xd42a bytes) 2024-09-18T01:03:55.740 INFO:teuthology.orchestra.run.smithi164.stdout:4332 read 0x51d3600 thru 0x51deb24 (0xb525 bytes) 2024-09-18T01:03:55.740 INFO:teuthology.orchestra.run.smithi164.stdout:4336 read 0x40a7400 thru 0x40ab58e (0x418f bytes) 2024-09-18T01:03:55.740 INFO:teuthology.orchestra.run.smithi164.stdout:4338 punch from 0xcf5400 to 0xcf9000, (0x3c00 bytes) 2024-09-18T01:03:55.752 INFO:teuthology.orchestra.run.smithi164.stdout:4339 write 0x76d8600 thru 0x76e1caa (0x96ab bytes) 2024-09-18T01:03:55.753 INFO:teuthology.orchestra.run.smithi164.stdout:4340 read 0x11e2a00 thru 0x11f1b5f (0xf160 bytes) 2024-09-18T01:03:55.754 INFO:teuthology.orchestra.run.smithi164.stdout:4341 read 0x652d600 thru 0x652e36c (0xd6d bytes) 2024-09-18T01:03:55.769 INFO:teuthology.orchestra.run.smithi164.stdout:4343 write 0x8fec800 thru 0x8feeb0e (0x230f bytes) 2024-09-18T01:03:55.770 INFO:teuthology.orchestra.run.smithi164.stdout:4344 read 0x55fac00 thru 0x55ffc5d (0x505e bytes) 2024-09-18T01:03:55.771 INFO:teuthology.orchestra.run.smithi164.stdout:4345 write 0x3ed8c00 thru 0x3ee5ac2 (0xcec3 bytes) 2024-09-18T01:03:55.781 INFO:teuthology.orchestra.run.smithi164.stdout:4346 trunc from 0x8feec00 to 0x7c0e00 2024-09-18T01:03:55.783 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:55.782+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:55.813 INFO:teuthology.orchestra.run.smithi164.stdout:4347 trunc from 0x7c0e00 to 0x77ee400 2024-09-18T01:03:55.824 INFO:teuthology.orchestra.run.smithi164.stdout:4348 punch from 0x3f72600 to 0x3f77000, (0x4a00 bytes) 2024-09-18T01:03:55.828 INFO:teuthology.orchestra.run.smithi164.stdout:4350 read 0x315b800 thru 0x316adc0 (0xf5c1 bytes) 2024-09-18T01:03:55.829 INFO:teuthology.orchestra.run.smithi164.stdout:4352 read 0x659f600 thru 0x65a7ba4 (0x85a5 bytes) 2024-09-18T01:03:55.829 INFO:teuthology.orchestra.run.smithi164.stdout:4354 punch from 0x66c3400 to 0x66d0600, (0xd200 bytes) 2024-09-18T01:03:55.834 INFO:teuthology.orchestra.run.smithi164.stdout:4357 trunc from 0x77ee400 to 0xc332800 2024-09-18T01:03:55.843 INFO:teuthology.orchestra.run.smithi164.stdout:4359 write 0x88ef600 thru 0x88f6ed4 (0x78d5 bytes) 2024-09-18T01:03:55.845 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:55.845 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:03:55.845 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:55.854 INFO:teuthology.orchestra.run.smithi164.stdout:4360 trunc from 0xc332800 to 0x3639e00 2024-09-18T01:03:55.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:55.884 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:55.883+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:55.895 INFO:teuthology.orchestra.run.smithi164.stdout:4361 write 0x8c31400 thru 0x8c32f15 (0x1b16 bytes) 2024-09-18T01:03:55.914 INFO:teuthology.orchestra.run.smithi164.stdout:4365 write 0xd11de00 thru 0xd12ad3c (0xcf3d bytes) 2024-09-18T01:03:55.916 INFO:teuthology.orchestra.run.smithi164.stdout:4367 write 0x2ff6800 thru 0x2ffbd03 (0x5504 bytes) 2024-09-18T01:03:55.916 INFO:teuthology.orchestra.run.smithi164.stdout:4369 read 0x8d25e00 thru 0x8d30e10 (0xb011 bytes) 2024-09-18T01:03:55.917 INFO:teuthology.orchestra.run.smithi164.stdout:4371 write 0xa5a7e00 thru 0xa5a7f73 (0x174 bytes) 2024-09-18T01:03:55.917 INFO:teuthology.orchestra.run.smithi164.stdout:4375 read 0xce8f800 thru 0xce94d72 (0x5573 bytes) 2024-09-18T01:03:55.917 INFO:teuthology.orchestra.run.smithi164.stdout:4380 punch from 0x79b5e00 to 0x79c4400, (0xe600 bytes) 2024-09-18T01:03:55.927 INFO:teuthology.orchestra.run.smithi164.stdout:4381 trunc from 0xd12ae00 to 0x4438a00 2024-09-18T01:03:55.947 INFO:teuthology.orchestra.run.smithi164.stdout:4385 trunc from 0x4438a00 to 0x5cf5e00 2024-09-18T01:03:55.973 INFO:teuthology.orchestra.run.smithi164.stdout:4386 write 0xb34e200 thru 0xb353f5c (0x5d5d bytes) 2024-09-18T01:03:55.974 INFO:teuthology.orchestra.run.smithi164.stdout:4389 punch from 0x9744400 to 0x9748000, (0x3c00 bytes) 2024-09-18T01:03:55.975 INFO:teuthology.orchestra.run.smithi164.stdout:4390 write 0x8842000 thru 0x8849ebb (0x7ebc bytes) 2024-09-18T01:03:55.980 INFO:teuthology.orchestra.run.smithi164.stdout:4392 trunc from 0xb354000 to 0xecdce00 2024-09-18T01:03:55.981 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:03:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:03:55.981 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:03:55.982 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:55.985 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:55.984+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:55.989 INFO:teuthology.orchestra.run.smithi164.stdout:4393 read 0xe2d7200 thru 0xe2d7db9 (0xbba bytes) 2024-09-18T01:03:55.989 INFO:teuthology.orchestra.run.smithi164.stdout:4400 read 0xc325000 thru 0xc334029 (0xf02a bytes) 2024-09-18T01:03:55.990 INFO:teuthology.orchestra.run.smithi164.stdout:4401 write 0x7765200 thru 0x7765749 (0x54a bytes) 2024-09-18T01:03:55.990 INFO:teuthology.orchestra.run.smithi164.stdout:4402 write 0x2ff3c00 thru 0x3000beb (0xcfec bytes) 2024-09-18T01:03:55.990 INFO:teuthology.orchestra.run.smithi164.stdout:4404 write 0x8ab8a00 thru 0x8ac4e84 (0xc485 bytes) 2024-09-18T01:03:55.991 INFO:teuthology.orchestra.run.smithi164.stdout:4406 write 0x6eb800 thru 0x6f1846 (0x6047 bytes) 2024-09-18T01:03:55.991 INFO:teuthology.orchestra.run.smithi164.stdout:4407 write 0x4f69c00 thru 0x4f7757c (0xd97d bytes) 2024-09-18T01:03:55.991 INFO:teuthology.orchestra.run.smithi164.stdout:4409 write 0x23db000 thru 0x23dc774 (0x1775 bytes) 2024-09-18T01:03:55.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:56.010 INFO:teuthology.orchestra.run.smithi164.stdout:4411 trunc from 0xecdce00 to 0x81d600 2024-09-18T01:03:56.036 INFO:teuthology.orchestra.run.smithi164.stdout:4413 read 0x433c00 thru 0x43952e (0x592f bytes) 2024-09-18T01:03:56.037 INFO:teuthology.orchestra.run.smithi164.stdout:4415 punch from 0x2d5a00 to 0x2e0600, (0xac00 bytes) 2024-09-18T01:03:56.040 INFO:teuthology.orchestra.run.smithi164.stdout:4416 trunc from 0x81d600 to 0x2741a00 2024-09-18T01:03:56.049 INFO:teuthology.orchestra.run.smithi164.stdout:4418 punch from 0x7b8e00 to 0x7c4000, (0xb200 bytes) 2024-09-18T01:03:56.054 INFO:teuthology.orchestra.run.smithi164.stdout:4419 punch from 0xbf1200 to 0xc00200, (0xf000 bytes) 2024-09-18T01:03:56.055 INFO:teuthology.orchestra.run.smithi164.stdout:4420 punch from 0x23df400 to 0x23e4600, (0x5200 bytes) 2024-09-18T01:03:56.055 INFO:teuthology.orchestra.run.smithi164.stdout:4422 clone 28 order 20 su 16384 sc 15 2024-09-18T01:03:56.096 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:56.085+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:56.156 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:03:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:03:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:03:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.418955617000002, 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:03:56.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.418935658999999, 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:03:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:03:56.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:03:56.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.418916717999998, 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:03:56.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:03:56.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:03:56.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.418895208999999, 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:03:56.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:03:56.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:03:56.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:03:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:03:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:03:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:03:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:03:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:03:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:03:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:03:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:03:56.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:03:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:03:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:03:56.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.188 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.418869481000002, 2024-09-18T01:03:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:03:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:03:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:03:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:03:56.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:03:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:03:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:03:56.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:03:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:03:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:03:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:03:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:03:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:03:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:03:56.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.199 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.410137622000001, 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:03:56.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:03:56.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:03:56.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.410114907999997, 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:03:56.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:03:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:03:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:03:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:03:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:03:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:03:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:03:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:03:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:03:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:03:56.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:03:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:03:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:03:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:03:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.410095777999999, 2024-09-18T01:03:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:03:56.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:03:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:03:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:03:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:03:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:03:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:03:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:03:56.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:03:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:03:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:03:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:03:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:03:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:03:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:03:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:03:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:03:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:03:56.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:03:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:03:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:03:56.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:03:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:03:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:03:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:03:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.410076932000003, 2024-09-18T01:03:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:03:56.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:03:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:03:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:03:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:03:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:03:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:03:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:03:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:03:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:03:56.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:03:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:03:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:03:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:03:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.410058053999997, 2024-09-18T01:03:56.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:03:56.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:03:56.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:03:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:03:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:03:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:03:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:03:56.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:03:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:03:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:03:56.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:03:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:03:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.410036189000003, 2024-09-18T01:03:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:03:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:03:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:56.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:03:56.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:03:56.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.410012553999998, 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:03:56.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:03:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:03:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:03:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:03:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:03:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:03:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:03:56.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:03:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:03:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:03:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:03:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:03:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:03:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:03:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:03:56.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:03:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:03:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:03:56.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409978903999999, 2024-09-18T01:03:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:03:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:03:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:03:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:03:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:03:56.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:03:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:03:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:03:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:03:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:03:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:03:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:03:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:03:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:03:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:03:56.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:03:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:03:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:03:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:03:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:03:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:03:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:03:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:03:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:03:56.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:03:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409951049, 2024-09-18T01:03:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:03:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:03:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:03:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:03:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:03:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:03:56.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:03:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:03:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:03:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:03:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:03:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:03:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:03:56.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:03:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:03:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:03:56.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:03:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:03:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:03:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:03:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:03:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:03:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:03:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:03:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409931225999998, 2024-09-18T01:03:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:03:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:03:56.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:03:56.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409910791999998, 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:03:56.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:56.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409888711999997, 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:03:56.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:03:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:03:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:03:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:03:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:03:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:03:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:03:56.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:03:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:03:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:03:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:03:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:03:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:03:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:03:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:03:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:03:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:03:56.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:03:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:03:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:03:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:03:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:03:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:03:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409866389999998, 2024-09-18T01:03:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:03:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:03:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:03:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:03:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:03:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:03:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:03:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:03:56.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:03:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:03:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:03:56.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:03:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:03:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:03:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:03:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:03:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:03:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:03:56.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:03:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:03:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:03:56.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:03:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:03:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:03:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:03:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409843993999999, 2024-09-18T01:03:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:03:56.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:03:56.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:03:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:03:56.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:03:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:03:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:03:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:03:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:03:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:03:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:03:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:03:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:03:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:03:56.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:03:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:03:56.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:03:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:03:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:03:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:03:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:03:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:03:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:03:56.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:03:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.409823574999997, 2024-09-18T01:03:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:03:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:03:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:03:56.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:03:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:03:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:03:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:03:56.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:03:56.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:56.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:03:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:03:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:03:56.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:03:56.366 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:56.367 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:56.368 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:03:56.368 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:56.346+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:56.414 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone27 from 0x2741a00 (overlap 0x6f4a00) to 0xb2000 2024-09-18T01:03:56.448 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:56.447+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:56.477 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:56.478 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:56.478 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:56.486 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:56.549 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:56.548+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:56.636 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:56.637 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:56.637 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:56.637 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:56.637 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:56.645 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:56.691 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:56.690+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:56.750 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:56.751 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:56.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:56.751 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:56.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.751 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.751 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:56.751 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.704331220999997, 2024-09-18T01:03:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:03:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.752 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.753 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:03:56.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.753 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:56.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:56.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:03:56.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:03:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.704320070000001, 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:03:56.758 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.761 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.704308240000003, 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:03:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:56.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:03:56.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.704291556999998, 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:03:56.768 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:56.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.773 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.704280320000002, 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.774 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.775 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.775 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.775 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.775 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:03:56.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.775 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:03:56.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:03:56.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.780 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.702958215000002, 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.781 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:03:56.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:03:56.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:03:56.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:03:56.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:03:56.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:03:56.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:56.788 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.702945499999998, 2024-09-18T01:03:56.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:03:56.789 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.789 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.789 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.789 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.789 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.789 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.790 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:03:56.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.790 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:03:56.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:03:56.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:03:56.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.794 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.702900516, 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:03:56.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:03:56.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:03:56.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:03:56.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:03:56.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:03:56.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:03:56.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:03:56.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:03:56.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:03:56.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:03:56.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:03:56.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:03:56.802 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.802 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.702872892000002, 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:03:56.803 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.804 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.804 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.804 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.804 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.804 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.804 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:03:56.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.805 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:56.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:03:56.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:03:56.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.699325512999998, 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:03:56.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:03:56.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:03:56.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:56.815 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.698965405999999, 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:03:56.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:03:56.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.698939293000002, 2024-09-18T01:03:56.820 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:56.821 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.824 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.696759905, 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.825 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:03:56.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.829 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.696750109999996, 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.830 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.696739331000003, 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:03:56.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:03:56.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.838 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.839 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:03:56.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:03:56.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.839 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.839 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:03:56.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.696719535, 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.841 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:03:56.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:03:56.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:03:56.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.694936015000003, 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.846 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:03:56.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.694909177, 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.851 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:03:56.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:03:56.855 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 54.694898629000001, 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.856 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:56.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:03:56.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.860 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 19.222971752999999, 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:03:56.861 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:03:56.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:03:56.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:56.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:56.866 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:56.867 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:56.749+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:56.868 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-18T01:03:56.868 INFO:teuthology.orchestra.run.smithi164.stdout:4426 trunc from 0x2741a00 to 0xc3fa000 2024-09-18T01:03:56.868 INFO:teuthology.orchestra.run.smithi164.stdout:4428 read 0x83aea00 thru 0x83ba6bf (0xbcc0 bytes) 2024-09-18T01:03:56.868 INFO:teuthology.orchestra.run.smithi164.stdout:4429 punch from 0x4f43e00 to 0x4f4e400, (0xa600 bytes) 2024-09-18T01:03:56.869 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:56.869 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:56.866+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:56.870 INFO:teuthology.orchestra.run.smithi164.stdout:4430 trunc from 0xc3fa000 to 0xe42d200 2024-09-18T01:03:56.885 INFO:teuthology.orchestra.run.smithi164.stdout:4431 write 0xd5cb000 thru 0xd5d3b97 (0x8b98 bytes) 2024-09-18T01:03:56.885 INFO:teuthology.orchestra.run.smithi164.stdout:4432 write 0xc3200 thru 0xc4683 (0x1484 bytes) 2024-09-18T01:03:56.886 INFO:teuthology.orchestra.run.smithi164.stdout:4433 punch from 0x31b2800 to 0x31b7400, (0x4c00 bytes) 2024-09-18T01:03:56.888 INFO:teuthology.orchestra.run.smithi164.stdout:4436 write 0xfdfa00 thru 0xfe7657 (0x7c58 bytes) 2024-09-18T01:03:56.890 INFO:teuthology.orchestra.run.smithi164.stdout:4437 read 0x3882600 thru 0x38840dc (0x1add bytes) 2024-09-18T01:03:56.896 INFO:teuthology.orchestra.run.smithi164.stdout:4440 trunc from 0xe42d200 to 0x9654c00 2024-09-18T01:03:56.952 INFO:teuthology.orchestra.run.smithi164.stdout:4441 write 0xcd47200 thru 0xcd47297 (0x98 bytes) 2024-09-18T01:03:56.952 INFO:teuthology.orchestra.run.smithi164.stdout:4445 punch from 0xa51c000 to 0xa528200, (0xc200 bytes) 2024-09-18T01:03:56.952 INFO:teuthology.orchestra.run.smithi164.stdout:4447 write 0xcc99e00 thru 0xcc9cd6a (0x2f6b bytes) 2024-09-18T01:03:56.953 INFO:teuthology.orchestra.run.smithi164.stdout:4450 read 0x3ff5a00 thru 0x3ff9286 (0x3887 bytes) 2024-09-18T01:03:56.953 INFO:teuthology.orchestra.run.smithi164.stdout:4451 read 0x136a200 thru 0x1371e04 (0x7c05 bytes) 2024-09-18T01:03:56.967 INFO:teuthology.orchestra.run.smithi164.stdout:4452 trunc from 0xcd47400 to 0x33ae00 2024-09-18T01:03:56.968 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:56.967+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:57.033 INFO:teuthology.orchestra.run.smithi164.stdout:4453 punch from 0x13bc00 to 0x147e00, (0xc200 bytes) 2024-09-18T01:03:57.035 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:57.036 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:57.036 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:57.036 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:57.036 INFO:teuthology.orchestra.run.smithi164.stdout:4455 punch from 0x2d3600 to 0x2d8000, (0x4a00 bytes) 2024-09-18T01:03:57.036 INFO:teuthology.orchestra.run.smithi164.stdout:4456 read 0x13cc00 thru 0x149e76 (0xd277 bytes) 2024-09-18T01:03:57.045 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:57.074 INFO:teuthology.orchestra.run.smithi164.stdout:4458 write 0xc806800 thru 0xc80d7c4 (0x6fc5 bytes) 2024-09-18T01:03:57.074 INFO:teuthology.orchestra.run.smithi164.stdout:4460 read 0x5bad000 thru 0x5bb6992 (0x9993 bytes) 2024-09-18T01:03:57.082 INFO:teuthology.orchestra.run.smithi164.stdout:4464 trunc from 0xc80d800 to 0x605f200 2024-09-18T01:03:57.087 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:57.086+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:57.113 INFO:teuthology.orchestra.run.smithi164.stdout:4467 trunc from 0x605f200 to 0xb444000 2024-09-18T01:03:57.123 INFO:teuthology.orchestra.run.smithi164.stdout:4468 read 0x4c58000 thru 0x4c581f5 (0x1f6 bytes) 2024-09-18T01:03:57.124 INFO:teuthology.orchestra.run.smithi164.stdout:4469 read 0xf5400 thru 0x101289 (0xbe8a bytes) 2024-09-18T01:03:57.128 INFO:teuthology.orchestra.run.smithi164.stdout:4471 write 0x716a600 thru 0x716e5a7 (0x3fa8 bytes) 2024-09-18T01:03:57.128 INFO:teuthology.orchestra.run.smithi164.stdout:4474 read 0xb429000 thru 0xb437eec (0xeeed bytes) 2024-09-18T01:03:57.139 INFO:teuthology.orchestra.run.smithi164.stdout:4475 trunc from 0xb444000 to 0x2379000 2024-09-18T01:03:57.170 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:57.168+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:57.180 INFO:teuthology.orchestra.run.smithi164.stdout:4476 write 0x7724800 thru 0x773245e (0xdc5f bytes) 2024-09-18T01:03:57.180 INFO:teuthology.orchestra.run.smithi164.stdout:4479 read 0x4504200 thru 0x450b873 (0x7674 bytes) 2024-09-18T01:03:57.180 INFO:teuthology.orchestra.run.smithi164.stdout:4482 punch from 0x5f4d400 to 0x5f55400, (0x8000 bytes) 2024-09-18T01:03:57.184 INFO:teuthology.orchestra.run.smithi164.stdout:4485 read 0x798c00 thru 0x7a4360 (0xb761 bytes) 2024-09-18T01:03:57.186 INFO:teuthology.orchestra.run.smithi164.stdout:4486 write 0x25e5600 thru 0x25e95fc (0x3ffd bytes) 2024-09-18T01:03:57.186 INFO:teuthology.orchestra.run.smithi164.stdout:4490 punch from 0xa9e000 to 0xaa8800, (0xa800 bytes) 2024-09-18T01:03:57.186 INFO:teuthology.orchestra.run.smithi164.stdout:4491 punch from 0x493600 to 0x49ee00, (0xb800 bytes) 2024-09-18T01:03:57.186 INFO:teuthology.orchestra.run.smithi164.stdout:4494 read 0x196200 thru 0x1a2f98 (0xcd99 bytes) 2024-09-18T01:03:57.192 INFO:teuthology.orchestra.run.smithi164.stdout:4495 trunc from 0x7732600 to 0xbae1200 2024-09-18T01:03:57.201 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:57.202 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:57.202 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:57.202 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:57.202 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:57.210 INFO:teuthology.orchestra.run.smithi164.stdout:4496 read 0xfb2a00 thru 0xfbd07e (0xa67f bytes) 2024-09-18T01:03:57.211 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:03:57.213 INFO:teuthology.orchestra.run.smithi164.stdout:4500 trunc from 0xbae1200 to 0xdccd600 2024-09-18T01:03:57.224 INFO:teuthology.orchestra.run.smithi164.stdout:4502 read 0xcd95000 thru 0xcd98fa8 (0x3fa9 bytes) 2024-09-18T01:03:57.225 INFO:teuthology.orchestra.run.smithi164.stdout:4503 read 0x5660e00 thru 0x5662af0 (0x1cf1 bytes) 2024-09-18T01:03:57.226 INFO:teuthology.orchestra.run.smithi164.stdout:4504 read 0x1b0a600 thru 0x1b16989 (0xc38a bytes) 2024-09-18T01:03:57.226 INFO:teuthology.orchestra.run.smithi164.stdout:4507 read 0x2365a00 thru 0x236a2eb (0x48ec bytes) 2024-09-18T01:03:57.226 INFO:teuthology.orchestra.run.smithi164.stdout:4508 punch from 0x4227800 to 0x422c800, (0x5000 bytes) 2024-09-18T01:03:57.230 INFO:teuthology.orchestra.run.smithi164.stdout:4509 read 0x13cdc00 thru 0x13d2c5f (0x5060 bytes) 2024-09-18T01:03:57.230 INFO:teuthology.orchestra.run.smithi164.stdout:4510 read 0xbd3ee00 thru 0xbd49d9b (0xaf9c bytes) 2024-09-18T01:03:57.230 INFO:teuthology.orchestra.run.smithi164.stdout:4512 read 0x9af000 thru 0x9bc990 (0xd991 bytes) 2024-09-18T01:03:57.231 INFO:teuthology.orchestra.run.smithi164.stdout:4513 read 0xe34600 thru 0xe43564 (0xef65 bytes) 2024-09-18T01:03:57.232 INFO:teuthology.orchestra.run.smithi164.stdout:4514 flatten 2024-09-18T01:03:57.245 INFO:teuthology.orchestra.run.smithi164.stdout:4518 read 0x4ee5000 thru 0x4ee585e (0x85f bytes) 2024-09-18T01:03:57.246 INFO:teuthology.orchestra.run.smithi164.stdout:4519 write 0x8a16c00 thru 0x8a1787e (0xc7f bytes) 2024-09-18T01:03:57.258 INFO:teuthology.orchestra.run.smithi164.stdout:4520 trunc from 0xdccd600 to 0x3235000 2024-09-18T01:03:57.290 INFO:teuthology.orchestra.run.smithi164.stdout:4521 read 0x2405800 thru 0x2407cd2 (0x24d3 bytes) 2024-09-18T01:03:57.291 INFO:teuthology.orchestra.run.smithi164.stdout:4522 trunc from 0x3235000 to 0x202d800 2024-09-18T01:03:57.296 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:57.295+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:4526 read 0xf2aa00 thru 0xf2fb56 (0x5157 bytes) 2024-09-18T01:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:4527 punch from 0x157c200 to 0x1583600, (0x7400 bytes) 2024-09-18T01:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:4530 punch from 0xab6c00 to 0xab8200, (0x1600 bytes) 2024-09-18T01:03:57.309 INFO:teuthology.orchestra.run.smithi164.stdout:4531 punch from 0x1837000 to 0x1846000, (0xf000 bytes) 2024-09-18T01:03:57.317 INFO:teuthology.orchestra.run.smithi164.stdout:4532 trunc from 0x202d800 to 0x894fe00 2024-09-18T01:03:57.328 INFO:teuthology.orchestra.run.smithi164.stdout:4533 punch from 0x13c1c00 to 0x13c5400, (0x3800 bytes) 2024-09-18T01:03:57.329 INFO:teuthology.orchestra.run.smithi164.stdout:4534 read 0x7f02e00 thru 0x7f03a97 (0xc98 bytes) 2024-09-18T01:03:57.330 INFO:teuthology.orchestra.run.smithi164.stdout:4538 trunc from 0x894fe00 to 0x9781c00 2024-09-18T01:03:57.340 INFO:teuthology.orchestra.run.smithi164.stdout:4539 write 0x4b30a00 thru 0x4b3d652 (0xcc53 bytes) 2024-09-18T01:03:57.341 INFO:teuthology.orchestra.run.smithi164.stdout:4540 read 0x1bcba00 thru 0x1bdafd0 (0xf5d1 bytes) 2024-09-18T01:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout:4541 write 0x901ca00 thru 0x90273a3 (0xa9a4 bytes) 2024-09-18T01:03:57.342 INFO:teuthology.orchestra.run.smithi164.stdout:4546 read 0x366cc00 thru 0x36760ff (0x9500 bytes) 2024-09-18T01:03:57.344 INFO:teuthology.orchestra.run.smithi164.stdout:4550 trunc from 0x9781c00 to 0xb7b6e00 2024-09-18T01:03:57.368 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:57.368 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:57.368 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.070835872000004, 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.369 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.370 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:03:57.371 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.372 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.070811755000001, 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:03:57.373 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.374 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:03:57.375 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.376 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.377 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.070732515000003, 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:03:57.379 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:03:57.380 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.381 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.070696296000001, 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:03:57.382 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:03:57.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.070667683000003, 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.387 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:03:57.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.070646818, 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.391 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.392 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:03:57.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:57.395 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.070029101000003, 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:03:57.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.399 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069999834000001, 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.400 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.402 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:03:57.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069933542999998, 2024-09-18T01:03:57.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:03:57.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069812775999999, 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:03:57.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:03:57.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069594330000001, 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.413 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:03:57.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069572968999999, 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:03:57.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.421 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069419029999999, 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.422 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:03:57.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069399312000002, 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:03:57.426 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.430 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069379001999998, 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.431 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:03:57.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.434 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069359116000001, 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.435 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069339157000002, 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:03:57.439 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.441 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.443 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069319159000003, 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:03:57.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:03:57.447 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069299491999999, 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.448 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:57.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:03:57.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 51.069279590000001, 2024-09-18T01:03:57.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:03:57.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.456 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.457 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.457 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.457 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.457 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:57.459 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:03:57.459 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:57.457+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:57.460 INFO:teuthology.orchestra.run.smithi164.stdout:4553 write 0x8a97400 thru 0x8aa59f1 (0xe5f2 bytes) 2024-09-18T01:03:57.460 INFO:teuthology.orchestra.run.smithi164.stdout:4554 trunc from 0xb7b6e00 to 0xe265400 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4556 read 0x22e3a00 thru 0x22ebbfa (0x81fb bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4557 write 0xcda3a00 thru 0xcdb35c7 (0xfbc8 bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4558 read 0x7be4600 thru 0x7be8129 (0x3b2a bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4560 read 0xbc20600 thru 0xbc219c0 (0x13c1 bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4562 read 0x9052800 thru 0x9054807 (0x2008 bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4564 punch from 0xbdf1800 to 0xbe00400, (0xec00 bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4565 read 0x6202e00 thru 0x62080ba (0x52bb bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4567 trunc from 0xe265400 to 0x9853a00 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4569 read 0x550a600 thru 0x5511b9e (0x759f bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4570 read 0x7671400 thru 0x767effb (0xdbfc bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4571 punch from 0x4406a00 to 0x440d400, (0x6a00 bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4572 punch from 0x72cbc00 to 0x72d3600, (0x7a00 bytes) 2024-09-18T01:03:57.461 INFO:teuthology.orchestra.run.smithi164.stdout:4574 write 0x9a4f000 thru 0x9a577c1 (0x87c2 bytes) 2024-09-18T01:03:57.475 INFO:teuthology.orchestra.run.smithi164.stdout:4575 write 0xafd8200 thru 0xafe7145 (0xef46 bytes) 2024-09-18T01:03:57.482 INFO:teuthology.orchestra.run.smithi164.stdout:4576 read 0x600f200 thru 0x6015a9a (0x689b bytes) 2024-09-18T01:03:57.492 INFO:teuthology.orchestra.run.smithi164.stdout:4577 trunc from 0xafe7200 to 0x27d5a00 2024-09-18T01:03:57.539 INFO:teuthology.orchestra.run.smithi164.stdout:4578 trunc from 0x27d5a00 to 0x7130000 2024-09-18T01:03:57.549 INFO:teuthology.orchestra.run.smithi164.stdout:4579 read 0x148a600 thru 0x148b6fd (0x10fe bytes) 2024-09-18T01:03:57.550 INFO:teuthology.orchestra.run.smithi164.stdout:4580 read 0x28a4200 thru 0x28aa194 (0x5f95 bytes) 2024-09-18T01:03:57.550 INFO:teuthology.orchestra.run.smithi164.stdout:4582 write 0x6bebc00 thru 0x6bf58cc (0x9ccd bytes) 2024-09-18T01:03:57.554 INFO:teuthology.orchestra.run.smithi164.stdout:4586 trunc from 0x7130000 to 0x3a20a00 2024-09-18T01:03:57.559 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:57.558+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:57.579 INFO:teuthology.orchestra.run.smithi164.stdout:4590 trunc from 0x3a20a00 to 0x33af600 2024-09-18T01:03:57.608 INFO:teuthology.orchestra.run.smithi164.stdout:4592 trunc from 0x33af600 to 0x8a2a800 2024-09-18T01:03:57.611 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:57.611 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:57.611 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:03:57.611 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:57.619 INFO:teuthology.orchestra.run.smithi164.stdout:4597 read 0x2b35400 thru 0x2b3579c (0x39d bytes) 2024-09-18T01:03:57.620 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:03:57.631 INFO:teuthology.orchestra.run.smithi164.stdout:4598 trunc from 0x8a2a800 to 0xd8be800 2024-09-18T01:03:57.642 INFO:teuthology.orchestra.run.smithi164.stdout:4599 write 0x4275a00 thru 0x42820be (0xc6bf bytes) 2024-09-18T01:03:57.655 INFO:teuthology.orchestra.run.smithi164.stdout:4601 trunc from 0xd8be800 to 0x86eb800 2024-09-18T01:03:57.659 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:57.658+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:57.674 INFO:teuthology.orchestra.run.smithi164.stdout:4602 read 0x8d600 thru 0x99ea3 (0xc8a4 bytes) 2024-09-18T01:03:57.676 INFO:teuthology.orchestra.run.smithi164.stdout:4603 read 0x2488400 thru 0x2492833 (0xa434 bytes) 2024-09-18T01:03:57.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:03:57.692 INFO:teuthology.orchestra.run.smithi164.stdout:4604 write 0xe4b5400 thru 0xe4c007b (0xac7c bytes) 2024-09-18T01:03:57.699 INFO:teuthology.orchestra.run.smithi164.stdout:4605 trunc from 0xe4c0200 to 0x87f3800 2024-09-18T01:03:57.720 INFO:teuthology.orchestra.run.smithi164.stdout:4607 punch from 0x8312600 to 0x8312800, (0x200 bytes) 2024-09-18T01:03:57.729 INFO:teuthology.orchestra.run.smithi164.stdout:4608 write 0x73ee00 thru 0x747896 (0x8a97 bytes) 2024-09-18T01:03:57.729 INFO:teuthology.orchestra.run.smithi164.stdout:4610 read 0x7902400 thru 0x7902c8d (0x88e bytes) 2024-09-18T01:03:57.735 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:57.735 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:03:57.735 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:03:57.735 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:03:57.735 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:57.740 INFO:teuthology.orchestra.run.smithi164.stdout:4612 write 0x9d12600 thru 0x9d1e696 (0xc097 bytes) 2024-09-18T01:03:57.741 INFO:teuthology.orchestra.run.smithi164.stdout:4615 read 0x30c4800 thru 0x30d0ba0 (0xc3a1 bytes) 2024-09-18T01:03:57.745 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:03:57.747 INFO:teuthology.orchestra.run.smithi164.stdout:4617 trunc from 0x9d1e800 to 0x578d400 2024-09-18T01:03:57.760 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:57.759+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:57.788 INFO:teuthology.orchestra.run.smithi164.stdout:4620 write 0xa510400 thru 0xa51cb28 (0xc729 bytes) 2024-09-18T01:03:57.789 INFO:teuthology.orchestra.run.smithi164.stdout:4621 read 0x7c34200 thru 0x7c37586 (0x3387 bytes) 2024-09-18T01:03:57.789 INFO:teuthology.orchestra.run.smithi164.stdout:4622 write 0x9998e00 thru 0x999c4a7 (0x36a8 bytes) 2024-09-18T01:03:57.819 INFO:teuthology.orchestra.run.smithi164.stdout:4624 write 0xe0b5000 thru 0xe0bfb63 (0xab64 bytes) 2024-09-18T01:03:57.820 INFO:teuthology.orchestra.run.smithi164.stdout:4625 punch from 0x2b4f200 to 0x2b50c00, (0x1a00 bytes) 2024-09-18T01:03:57.822 INFO:teuthology.orchestra.run.smithi164.stdout:4629 write 0xc56a800 thru 0xc56db7e (0x337f bytes) 2024-09-18T01:03:57.822 INFO:teuthology.orchestra.run.smithi164.stdout:4631 read 0x6df3000 thru 0x6dfe698 (0xb699 bytes) 2024-09-18T01:03:57.822 INFO:teuthology.orchestra.run.smithi164.stdout:4637 trunc from 0xe0bfc00 to 0xe259400 2024-09-18T01:03:57.822 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:03:57.823 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:03:57.842 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:03:57.842 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:03:57.846 INFO:teuthology.orchestra.run.smithi164.stdout:4640 trunc from 0xe259400 to 0x9ad5200 2024-09-18T01:03:57.862 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:03:57.862 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:03:57.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:03:57.862 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:03:57.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.862 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:03:57.862 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 85.485721940000005, 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:03:57.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:03:57.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:03:57.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.869 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 84.487710634999999, 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:03:57.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:03:57.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:03:57.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:57.876 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 83.480033683000002, 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:03:57.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:03:57.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:03:57.883 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.382175732999997, 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:03:57.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:57.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.890 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.526094721000007, 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:03:57.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:03:57.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:03:57.896 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:03:57.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.896 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.897 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.525725425000005, 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:03:57.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.902 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.903 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.523986579999999, 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.904 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:03:57.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.523914689999998, 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:03:57.909 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:03:57.911 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.436048679999999, 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.914 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.915 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:03:57.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.917 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:03:57.919 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.429857420999994, 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.920 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.921 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.922 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:03:57.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.924 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.925 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.428327736, 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:03:57.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.928 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.929 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:57.930 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:03:57.930 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:03:57.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.930 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.932 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.933 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.427948047000001, 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.934 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:03:57.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:03:57.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:03:57.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.939 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.412449644000006, 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.940 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:03:57.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:03:57.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.412242884999998, 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.946 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.947 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:03:57.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.411825128000004, 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:03:57.952 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.956 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.411615889999993, 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:03:57.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.411551955999997, 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:57.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:03:57.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:57.966 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.410803751000003, 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:03:57.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.151864946000003, 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.971 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:03:57.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.974 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:03:57.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:57.977 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.151001328000007, 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:03:57.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:03:57.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:03:57.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:03:57.983 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:03:57.984 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:57.860+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:57.985 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:03:57.985 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:03:57.986 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:03:57.987 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:03:57.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:03:57.988 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:03:57.988 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:4641 write 0x8c38e00 thru 0x8c4672c (0xd92d bytes) 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:4642 read 0x7efc00 thru 0x7f9397 (0x9798 bytes) 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:4644 write 0xcb17c00 thru 0xcb25be8 (0xdfe9 bytes) 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:4645 punch from 0x62d8200 to 0x62e3e00, (0xbc00 bytes) 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:4647 punch from 0x1c4600 to 0x1c5200, (0xc00 bytes) 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:4648 write 0xee59e00 thru 0xee665af (0xc7b0 bytes) 2024-09-18T01:03:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:4649 read 0x528e600 thru 0x52976fc (0x90fd bytes) 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4651 write 0x783b600 thru 0x7844e42 (0x9843 bytes) 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4652 write 0xbef4800 thru 0xbeffe3d (0xb63e bytes) 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4656 read 0xb121200 thru 0xb130436 (0xf237 bytes) 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4657 write 0x2e0ce00 thru 0x2e0ebc0 (0x1dc1 bytes) 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4659 trunc from 0xee66600 to 0xd3eb600 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4664 read 0xb6a1400 thru 0xb6a8017 (0x6c18 bytes) 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4666 write 0x819e600 thru 0x81a6449 (0x7e4a bytes) 2024-09-18T01:03:57.990 INFO:teuthology.orchestra.run.smithi164.stdout:4668 trunc from 0xd3eb600 to 0x602c000 2024-09-18T01:03:57.991 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:03:57.991 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:57.983+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:57.998 INFO:teuthology.orchestra.run.smithi164.stdout:4669 read 0x51a2400 thru 0x51ae94a (0xc54b bytes) 2024-09-18T01:03:57.999 INFO:teuthology.orchestra.run.smithi164.stdout:4670 punch from 0x5673800 to 0x567b600, (0x7e00 bytes) 2024-09-18T01:03:58.000 INFO:teuthology.orchestra.run.smithi164.stdout:4673 read 0x399f800 thru 0x39ab261 (0xba62 bytes) 2024-09-18T01:03:58.000 INFO:teuthology.orchestra.run.smithi164.stdout:4674 write 0x285ea00 thru 0x2869e3f (0xb440 bytes) 2024-09-18T01:03:58.000 INFO:teuthology.orchestra.run.smithi164.stdout:4675 read 0x464d000 thru 0x464e8c6 (0x18c7 bytes) 2024-09-18T01:03:58.018 INFO:teuthology.orchestra.run.smithi164.stdout:4676 write 0x6a6dc00 thru 0x6a7ba30 (0xde31 bytes) 2024-09-18T01:03:58.018 INFO:teuthology.orchestra.run.smithi164.stdout:4677 write 0x5bdb600 thru 0x5be5391 (0x9d92 bytes) 2024-09-18T01:03:58.044 INFO:teuthology.orchestra.run.smithi164.stdout:4681 write 0xd463a00 thru 0xd46679b (0x2d9c bytes) 2024-09-18T01:03:58.045 INFO:teuthology.orchestra.run.smithi164.stdout:4684 punch from 0x948e200 to 0x949d600, (0xf400 bytes) 2024-09-18T01:03:58.046 INFO:teuthology.orchestra.run.smithi164.stdout:4685 read 0x2e8400 thru 0x2e959c (0x119d bytes) 2024-09-18T01:03:58.047 INFO:teuthology.orchestra.run.smithi164.stdout:4686 read 0x7e54400 thru 0x7e59843 (0x5444 bytes) 2024-09-18T01:03:58.047 INFO:teuthology.orchestra.run.smithi164.stdout:4690 punch from 0xa16be00 to 0xa177600, (0xb800 bytes) 2024-09-18T01:03:58.047 INFO:teuthology.orchestra.run.smithi164.stdout:4691 punch from 0x59ba200 to 0x59bb200, (0x1000 bytes) 2024-09-18T01:03:58.047 INFO:teuthology.orchestra.run.smithi164.stdout:4693 write 0xb9d6800 thru 0xb9dbf4a (0x574b bytes) 2024-09-18T01:03:58.047 INFO:teuthology.orchestra.run.smithi164.stdout:4694 punch from 0x71f8800 to 0x7203800, (0xb000 bytes) 2024-09-18T01:03:58.047 INFO:teuthology.orchestra.run.smithi164.stdout:4696 punch from 0x5e7cc00 to 0x5e82a00, (0x5e00 bytes) 2024-09-18T01:03:58.056 INFO:teuthology.orchestra.run.smithi164.stdout:4697 trunc from 0xd466800 to 0x5a77e00 2024-09-18T01:03:58.085 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:58.084+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:58.091 INFO:teuthology.orchestra.run.smithi164.stdout:4698 write 0x34be00 thru 0x3595ee (0xd7ef bytes) 2024-09-18T01:03:58.092 INFO:teuthology.orchestra.run.smithi164.stdout:4700 punch from 0x120aa00 to 0x120ba00, (0x1000 bytes) 2024-09-18T01:03:58.094 INFO:teuthology.orchestra.run.smithi164.stdout:4702 read 0x3ca2e00 thru 0x3ca5457 (0x2658 bytes) 2024-09-18T01:03:58.099 INFO:teuthology.orchestra.run.smithi164.stdout:4703 trunc from 0x5a77e00 to 0x9c87a00 2024-09-18T01:03:58.108 INFO:teuthology.orchestra.run.smithi164.stdout:4704 write 0x2702800 thru 0x270bc5f (0x9460 bytes) 2024-09-18T01:03:58.108 INFO:teuthology.orchestra.run.smithi164.stdout:4705 read 0x8c43200 thru 0x8c4418f (0xf90 bytes) 2024-09-18T01:03:58.109 INFO:teuthology.orchestra.run.smithi164.stdout:4706 punch from 0x3e60000 to 0x3e63400, (0x3400 bytes) 2024-09-18T01:03:58.111 INFO:teuthology.orchestra.run.smithi164.stdout:4708 read 0x2005200 thru 0x2011825 (0xc626 bytes) 2024-09-18T01:03:58.128 INFO:teuthology.orchestra.run.smithi164.stdout:4710 write 0xe153200 thru 0xe1604b4 (0xd2b5 bytes) 2024-09-18T01:03:58.128 INFO:teuthology.orchestra.run.smithi164.stdout:4714 read 0xb85a000 thru 0xb863053 (0x9054 bytes) 2024-09-18T01:03:58.129 INFO:teuthology.orchestra.run.smithi164.stdout:4717 read 0x4f1e00 thru 0x4f9d66 (0x7f67 bytes) 2024-09-18T01:03:58.129 INFO:teuthology.orchestra.run.smithi164.stdout:4718 read 0xad4f800 thru 0xad50a16 (0x1217 bytes) 2024-09-18T01:03:58.129 INFO:teuthology.orchestra.run.smithi164.stdout:4723 punch from 0x2d86c00 to 0x2d87a00, (0xe00 bytes) 2024-09-18T01:03:58.130 INFO:teuthology.orchestra.run.smithi164.stdout:4726 write 0x5536e00 thru 0x553f272 (0x8473 bytes) 2024-09-18T01:03:58.130 INFO:teuthology.orchestra.run.smithi164.stdout:4729 read 0x1970800 thru 0x197df33 (0xd734 bytes) 2024-09-18T01:03:58.130 INFO:teuthology.orchestra.run.smithi164.stdout:4730 write 0x8201e00 thru 0x8205c67 (0x3e68 bytes) 2024-09-18T01:03:58.130 INFO:teuthology.orchestra.run.smithi164.stdout:4731 read 0x1a77200 thru 0x1a7baba (0x48bb bytes) 2024-09-18T01:03:58.131 INFO:teuthology.orchestra.run.smithi164.stdout:4733 read 0x93b0600 thru 0x93b5f1e (0x591f bytes) 2024-09-18T01:03:58.131 INFO:teuthology.orchestra.run.smithi164.stdout:4734 write 0x9fe7e00 thru 0x9febcfd (0x3efe bytes) 2024-09-18T01:03:58.131 INFO:teuthology.orchestra.run.smithi164.stdout:4736 punch from 0xc56c800 to 0xc577c00, (0xb400 bytes) 2024-09-18T01:03:58.132 INFO:teuthology.orchestra.run.smithi164.stdout:4737 punch from 0x8954200 to 0x8955e00, (0x1c00 bytes) 2024-09-18T01:03:58.132 INFO:teuthology.orchestra.run.smithi164.stdout:4741 write 0xd827000 thru 0xd8302ca (0x92cb bytes) 2024-09-18T01:03:58.132 INFO:teuthology.orchestra.run.smithi164.stdout:4744 write 0x8530a00 thru 0x85345be (0x3bbf bytes) 2024-09-18T01:03:58.146 INFO:teuthology.orchestra.run.smithi164.stdout:4745 trunc from 0xe160600 to 0x1c6b400 2024-09-18T01:03:58.161 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:58.161 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:58.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:03:58.213 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:58.212+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:58.216 INFO:teuthology.orchestra.run.smithi164.stdout:4746 write 0x780ec00 thru 0x7813de5 (0x51e6 bytes) 2024-09-18T01:03:58.216 INFO:teuthology.orchestra.run.smithi164.stdout:4747 trunc from 0x7813e00 to 0x7a4f400 2024-09-18T01:03:58.233 INFO:teuthology.orchestra.run.smithi164.stdout:4748 punch from 0x27a3600 to 0x27af400, (0xbe00 bytes) 2024-09-18T01:03:58.235 INFO:teuthology.orchestra.run.smithi164.stdout:4749 read 0x2a3b200 thru 0x2a46566 (0xb367 bytes) 2024-09-18T01:03:58.236 INFO:teuthology.orchestra.run.smithi164.stdout:4750 trunc from 0x7a4f400 to 0x6d86a00 2024-09-18T01:03:58.258 INFO:teuthology.orchestra.run.smithi164.stdout:4751 read 0x2afc000 thru 0x2afc54d (0x54e bytes) 2024-09-18T01:03:58.260 INFO:teuthology.orchestra.run.smithi164.stdout:4752 read 0x3dab200 thru 0x3dba38f (0xf190 bytes) 2024-09-18T01:03:58.260 INFO:teuthology.orchestra.run.smithi164.stdout:4754 punch from 0x2922200 to 0x2930a00, (0xe800 bytes) 2024-09-18T01:03:58.261 INFO:teuthology.orchestra.run.smithi164.stdout:4755 write 0x1964e00 thru 0x19698d6 (0x4ad7 bytes) 2024-09-18T01:03:58.261 INFO:teuthology.orchestra.run.smithi164.stdout:4759 read 0x12a8400 thru 0x12b737c (0xef7d bytes) 2024-09-18T01:03:58.262 INFO:teuthology.orchestra.run.smithi164.stdout:4760 punch from 0x3a94e00 to 0x3a9e600, (0x9800 bytes) 2024-09-18T01:03:58.287 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:58.286+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:58.289 INFO:teuthology.orchestra.run.smithi164.stdout:4761 write 0xa783000 thru 0xa78d734 (0xa735 bytes) 2024-09-18T01:03:58.289 INFO:teuthology.orchestra.run.smithi164.stdout:4762 read 0x12a0c00 thru 0x12a8e3e (0x823f bytes) 2024-09-18T01:03:58.290 INFO:teuthology.orchestra.run.smithi164.stdout:4763 punch from 0x22bb600 to 0x22c2e00, (0x7800 bytes) 2024-09-18T01:03:58.291 INFO:teuthology.orchestra.run.smithi164.stdout:4764 read 0x6116400 thru 0x6125e88 (0xfa89 bytes) 2024-09-18T01:03:58.291 INFO:teuthology.orchestra.run.smithi164.stdout:4765 read 0x6732800 thru 0x6732a29 (0x22a bytes) 2024-09-18T01:03:58.291 INFO:teuthology.orchestra.run.smithi164.stdout:4766 read 0xa06200 thru 0xa1443e (0xe23f bytes) 2024-09-18T01:03:58.295 INFO:teuthology.orchestra.run.smithi164.stdout:4767 read 0x5dd7400 thru 0x5de498b (0xd58c bytes) 2024-09-18T01:03:58.296 INFO:teuthology.orchestra.run.smithi164.stdout:4768 write 0xa95000 thru 0xa9b228 (0x6229 bytes) 2024-09-18T01:03:58.296 INFO:teuthology.orchestra.run.smithi164.stdout:4769 write 0xa30e600 thru 0xa31b14f (0xcb50 bytes) 2024-09-18T01:03:58.296 INFO:teuthology.orchestra.run.smithi164.stdout:4770 read 0x302f600 thru 0x3030df3 (0x17f4 bytes) 2024-09-18T01:03:58.297 INFO:teuthology.orchestra.run.smithi164.stdout:4772 write 0x807e200 thru 0x80889fd (0xa7fe bytes) 2024-09-18T01:03:58.297 INFO:teuthology.orchestra.run.smithi164.stdout:4774 write 0x6ffde00 thru 0x7000874 (0x2a75 bytes) 2024-09-18T01:03:58.297 INFO:teuthology.orchestra.run.smithi164.stdout:4775 read 0x8c2f800 thru 0x8c32102 (0x2903 bytes) 2024-09-18T01:03:58.297 INFO:teuthology.orchestra.run.smithi164.stdout:4777 write 0x2166a00 thru 0x216c1f3 (0x57f4 bytes) 2024-09-18T01:03:58.306 INFO:teuthology.orchestra.run.smithi164.stdout:4781 trunc from 0xa78d800 to 0x2596000 2024-09-18T01:03:58.337 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:58.337 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:58.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:03:58.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:03:58.372 INFO:teuthology.orchestra.run.smithi164.stdout:4782 write 0x6021200 thru 0x6028772 (0x7573 bytes) 2024-09-18T01:03:58.392 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:58.391+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:58.395 INFO:teuthology.orchestra.run.smithi164.stdout:4783 write 0xb44ea00 thru 0xb45dc2e (0xf22f bytes) 2024-09-18T01:03:58.395 INFO:teuthology.orchestra.run.smithi164.stdout:4784 punch from 0x4198600 to 0x419e800, (0x6200 bytes) 2024-09-18T01:03:58.397 INFO:teuthology.orchestra.run.smithi164.stdout:4788 read 0x5924600 thru 0x592d6e3 (0x90e4 bytes) 2024-09-18T01:03:58.399 INFO:teuthology.orchestra.run.smithi164.stdout:4790 trunc from 0xb45de00 to 0xa30fa00 2024-09-18T01:03:58.429 INFO:teuthology.orchestra.run.smithi164.stdout:4792 write 0xa020800 thru 0xa02f101 (0xe902 bytes) 2024-09-18T01:03:58.430 INFO:teuthology.orchestra.run.smithi164.stdout:4794 read 0x5c37000 thru 0x5c3dc06 (0x6c07 bytes) 2024-09-18T01:03:58.430 INFO:teuthology.orchestra.run.smithi164.stdout:4795 read 0x960f000 thru 0x961a741 (0xb742 bytes) 2024-09-18T01:03:58.430 INFO:teuthology.orchestra.run.smithi164.stdout:4796 write 0x6393000 thru 0x639ae6f (0x7e70 bytes) 2024-09-18T01:03:58.430 INFO:teuthology.orchestra.run.smithi164.stdout:4798 read 0x2d20400 thru 0x2d2f9ae (0xf5af bytes) 2024-09-18T01:03:58.431 INFO:teuthology.orchestra.run.smithi164.stdout:4799 write 0x542b800 thru 0x542db1c (0x231d bytes) 2024-09-18T01:03:58.431 INFO:teuthology.orchestra.run.smithi164.stdout:4800 trunc from 0xa30fa00 to 0x9715600 2024-09-18T01:03:58.459 INFO:teuthology.orchestra.run.smithi164.stdout:4801 punch from 0x30b6600 to 0x30c0000, (0x9a00 bytes) 2024-09-18T01:03:58.461 INFO:teuthology.orchestra.run.smithi164.stdout:4804 read 0x9068800 thru 0x906c3b4 (0x3bb5 bytes) 2024-09-18T01:03:58.477 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:58.477 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:58.477 INFO:teuthology.orchestra.run.smithi164.stdout:4805 write 0xb9f2200 thru 0xb9fbc67 (0x9a68 bytes) 2024-09-18T01:03:58.478 INFO:teuthology.orchestra.run.smithi164.stdout:4806 punch from 0x1ac1e00 to 0x1ace600, (0xc800 bytes) 2024-09-18T01:03:58.488 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:03:58.490 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:58.489+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:58.590 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:58.590+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:58.600 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:58.601 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:58.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:03:58.691 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:58.691+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:58.717 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:58.717 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:58.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:03:58.753 INFO:teuthology.orchestra.run.smithi164.stdout:4808 write 0xbf17800 thru 0xbf26e3a (0xf63b bytes) 2024-09-18T01:03:58.753 INFO:teuthology.orchestra.run.smithi164.stdout:4809 read 0x2b05400 thru 0x2b0760c (0x220d bytes) 2024-09-18T01:03:58.753 INFO:teuthology.orchestra.run.smithi164.stdout:4811 punch from 0x4875c00 to 0x487f400, (0x9800 bytes) 2024-09-18T01:03:58.753 INFO:teuthology.orchestra.run.smithi164.stdout:4812 read 0x2664c00 thru 0x266a55a (0x595b bytes) 2024-09-18T01:03:58.754 INFO:teuthology.orchestra.run.smithi164.stdout:4813 write 0x662e400 thru 0x663a7da (0xc3db bytes) 2024-09-18T01:03:58.754 INFO:teuthology.orchestra.run.smithi164.stdout:4814 read 0x36f1000 thru 0x36fd96f (0xc970 bytes) 2024-09-18T01:03:58.760 INFO:teuthology.orchestra.run.smithi164.stdout:4817 trunc from 0xbf27000 to 0x7170000 2024-09-18T01:03:58.787 INFO:teuthology.orchestra.run.smithi164.stdout:4819 read 0x1aca200 thru 0x1acd798 (0x3599 bytes) 2024-09-18T01:03:58.788 INFO:teuthology.orchestra.run.smithi164.stdout:4824 read 0x5147a00 thru 0x51523f8 (0xa9f9 bytes) 2024-09-18T01:03:58.789 INFO:teuthology.orchestra.run.smithi164.stdout:4825 punch from 0x47a3800 to 0x47b2400, (0xec00 bytes) 2024-09-18T01:03:58.793 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:03:58.792+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:03:58.809 INFO:teuthology.orchestra.run.smithi164.stdout:4827 write 0xa74a200 thru 0xa74e655 (0x4456 bytes) 2024-09-18T01:03:58.809 INFO:teuthology.orchestra.run.smithi164.stdout:4828 write 0x9f8d800 thru 0x9f8ef81 (0x1782 bytes) 2024-09-18T01:03:58.810 INFO:teuthology.orchestra.run.smithi164.stdout:4830 read 0x6954000 thru 0x695f729 (0xb72a bytes) 2024-09-18T01:03:58.810 INFO:teuthology.orchestra.run.smithi164.stdout:4831 punch from 0x5703400 to 0x570a800, (0x7400 bytes) 2024-09-18T01:03:58.811 INFO:teuthology.orchestra.run.smithi164.stdout:4834 punch from 0x395f400 to 0x3967c00, (0x8800 bytes) 2024-09-18T01:03:58.811 INFO:teuthology.orchestra.run.smithi164.stdout:4835 write 0x5391a00 thru 0x5392142 (0x743 bytes) 2024-09-18T01:03:58.811 INFO:teuthology.orchestra.run.smithi164.stdout:4837 write 0x56ae800 thru 0x56bceb4 (0xe6b5 bytes) 2024-09-18T01:03:58.812 INFO:teuthology.orchestra.run.smithi164.stdout:4838 write 0x1284c00 thru 0x1287230 (0x2631 bytes) 2024-09-18T01:03:58.812 INFO:teuthology.orchestra.run.smithi164.stdout:4841 write 0x8a6e600 thru 0x8a6f9b7 (0x13b8 bytes) 2024-09-18T01:03:58.812 INFO:teuthology.orchestra.run.smithi164.stdout:4843 read 0x62aa00 thru 0x639bc4 (0xf1c5 bytes) 2024-09-18T01:03:58.814 INFO:teuthology.orchestra.run.smithi164.stdout:4844 punch from 0x76eb000 to 0x76ef400, (0x4400 bytes) 2024-09-18T01:03:58.820 INFO:teuthology.orchestra.run.smithi164.stdout:4847 trunc from 0xa74e800 to 0x5525800 2024-09-18T01:03:58.852 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:58.852 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:58.857 INFO:teuthology.orchestra.run.smithi164.stdout:4849 read 0x56be00 thru 0x56e093 (0x2294 bytes) 2024-09-18T01:03:58.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:03:58.862 INFO:teuthology.orchestra.run.smithi164.stdout:4850 trunc from 0x5525800 to 0x1d73c00 2024-09-18T01:03:58.894 INFO:teuthology.orchestra.run.smithi164.stdout:4851 read 0x1cafc00 thru 0x1caff86 (0x387 bytes) 2024-09-18T01:03:58.905 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:58.904+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:58.916 INFO:teuthology.orchestra.run.smithi164.stdout:4852 write 0xbae0400 thru 0xbaebf33 (0xbb34 bytes) 2024-09-18T01:03:58.939 INFO:teuthology.orchestra.run.smithi164.stdout:4855 write 0xe343600 thru 0xe3505cd (0xcfce bytes) 2024-09-18T01:03:58.940 INFO:teuthology.orchestra.run.smithi164.stdout:4856 read 0x22e3e00 thru 0x22edb31 (0x9d32 bytes) 2024-09-18T01:03:58.941 INFO:teuthology.orchestra.run.smithi164.stdout:4859 trunc from 0xe350600 to 0xcc28e00 2024-09-18T01:03:58.949 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:03:58.949 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:03:58.949 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-18T01:03:58.949 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:03:58.949 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:03:58.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-18T01:03:58.975 INFO:teuthology.orchestra.run.smithi164.stdout:4861 punch from 0xc054800 to 0xc058400, (0x3c00 bytes) 2024-09-18T01:03:58.978 INFO:teuthology.orchestra.run.smithi164.stdout:4862 trunc from 0xcc28e00 to 0xca16e00 2024-09-18T01:03:58.990 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:58.991 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:58.993 INFO:teuthology.orchestra.run.smithi164.stdout:4864 write 0x4dfe00 thru 0x4efab0 (0xfcb1 bytes) 2024-09-18T01:03:58.995 INFO:teuthology.orchestra.run.smithi164.stdout:4866 read 0x36a4a00 thru 0x36a5936 (0xf37 bytes) 2024-09-18T01:03:58.995 INFO:teuthology.orchestra.run.smithi164.stdout:4868 read 0x11e8800 thru 0x11f15b1 (0x8db2 bytes) 2024-09-18T01:03:58.995 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:58.994+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:58.996 INFO:teuthology.orchestra.run.smithi164.stdout:4869 read 0xa033600 thru 0xa034ef7 (0x18f8 bytes) 2024-09-18T01:03:58.996 INFO:teuthology.orchestra.run.smithi164.stdout:4873 write 0x9ac4200 thru 0x9ad06d4 (0xc4d5 bytes) 2024-09-18T01:03:58.996 INFO:teuthology.orchestra.run.smithi164.stdout:4874 read 0x14ac600 thru 0x14afae1 (0x34e2 bytes) 2024-09-18T01:03:58.996 INFO:teuthology.orchestra.run.smithi164.stdout:4875 write 0xc093000 thru 0xc0a2abe (0xfabf bytes) 2024-09-18T01:03:58.997 INFO:teuthology.orchestra.run.smithi164.stdout:4879 write 0x4bb3e00 thru 0x4bc36ce (0xf8cf bytes) 2024-09-18T01:03:58.997 INFO:teuthology.orchestra.run.smithi164.stdout:4880 read 0x8ca3a00 thru 0x8ca59f8 (0x1ff9 bytes) 2024-09-18T01:03:58.997 INFO:teuthology.orchestra.run.smithi164.stdout:4881 write 0x38b4000 thru 0x38b861b (0x461c bytes) 2024-09-18T01:03:58.997 INFO:teuthology.orchestra.run.smithi164.stdout:4882 read 0x5b67e00 thru 0x5b69b71 (0x1d72 bytes) 2024-09-18T01:03:58.998 INFO:teuthology.orchestra.run.smithi164.stdout:4883 trunc from 0xca16e00 to 0xe887400 2024-09-18T01:03:59.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:03:59.030 INFO:teuthology.orchestra.run.smithi164.stdout:4884 read 0xdbee600 thru 0xdbf8121 (0x9b22 bytes) 2024-09-18T01:03:59.046 INFO:teuthology.orchestra.run.smithi164.stdout:4886 trunc from 0xe887400 to 0x829200 2024-09-18T01:03:59.079 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:03:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:03:59.079 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:03:59.096 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:59.095+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:59.100 INFO:teuthology.orchestra.run.smithi164.stdout:4888 write 0x3bb5200 thru 0x3bc2463 (0xd264 bytes) 2024-09-18T01:03:59.128 INFO:teuthology.orchestra.run.smithi164.stdout:4889 write 0xb0c4a00 thru 0xb0c7d03 (0x3304 bytes) 2024-09-18T01:03:59.130 INFO:teuthology.orchestra.run.smithi164.stdout:4890 punch from 0x49d6800 to 0x49d7400, (0xc00 bytes) 2024-09-18T01:03:59.138 INFO:teuthology.orchestra.run.smithi164.stdout:4891 trunc from 0xb0c7e00 to 0x5e82200 2024-09-18T01:03:59.156 INFO:teuthology.orchestra.run.smithi164.stdout:4892 write 0x3f40200 thru 0x3f42117 (0x1f18 bytes) 2024-09-18T01:03:59.158 INFO:teuthology.orchestra.run.smithi164.stdout:4894 write 0x59a1c00 thru 0x59a2bf3 (0xff4 bytes) 2024-09-18T01:03:59.158 INFO:teuthology.orchestra.run.smithi164.stdout:4898 read 0x37c2800 thru 0x37d0db1 (0xe5b2 bytes) 2024-09-18T01:03:59.167 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:59.168 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:59.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:03:59.179 INFO:teuthology.orchestra.run.smithi164.stdout:4900 write 0xb264800 thru 0xb26bbe2 (0x73e3 bytes) 2024-09-18T01:03:59.197 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:59.196+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:59.197 INFO:teuthology.orchestra.run.smithi164.stdout:4902 write 0xd1ffc00 thru 0xd2055a1 (0x59a2 bytes) 2024-09-18T01:03:59.198 INFO:teuthology.orchestra.run.smithi164.stdout:4903 read 0xc88e00 thru 0xc8d098 (0x4299 bytes) 2024-09-18T01:03:59.209 INFO:teuthology.orchestra.run.smithi164.stdout:4904 trunc from 0xd205600 to 0x3972800 2024-09-18T01:03:59.249 INFO:teuthology.orchestra.run.smithi164.stdout:4905 write 0x809f600 thru 0x80a20e4 (0x2ae5 bytes) 2024-09-18T01:03:59.257 INFO:teuthology.orchestra.run.smithi164.stdout:4908 trunc from 0x80a2200 to 0xe0f0600 2024-09-18T01:03:59.271 INFO:teuthology.orchestra.run.smithi164.stdout:4911 punch from 0xaf13600 to 0xaf22800, (0xf200 bytes) 2024-09-18T01:03:59.276 INFO:teuthology.orchestra.run.smithi164.stdout:4912 write 0x5c58a00 thru 0x5c67e80 (0xf481 bytes) 2024-09-18T01:03:59.276 INFO:teuthology.orchestra.run.smithi164.stdout:4913 write 0x4244800 thru 0x424d947 (0x9148 bytes) 2024-09-18T01:03:59.276 INFO:teuthology.orchestra.run.smithi164.stdout:4914 read 0x3b18e00 thru 0x3b25680 (0xc881 bytes) 2024-09-18T01:03:59.277 INFO:teuthology.orchestra.run.smithi164.stdout:4916 write 0x19b9200 thru 0x19bcb61 (0x3962 bytes) 2024-09-18T01:03:59.277 INFO:teuthology.orchestra.run.smithi164.stdout:4918 read 0x31b9800 thru 0x31b9832 (0x33 bytes) 2024-09-18T01:03:59.277 INFO:teuthology.orchestra.run.smithi164.stdout:4919 write 0xc974800 thru 0xc97e16f (0x9970 bytes) 2024-09-18T01:03:59.283 INFO:teuthology.orchestra.run.smithi164.stdout:4920 trunc from 0xe0f0600 to 0x809b400 2024-09-18T01:03:59.298 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:03:59.297+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:03:59.305 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:59.305 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:59.315 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:03:59.334 INFO:teuthology.orchestra.run.smithi164.stdout:4921 trunc from 0x809b400 to 0xaa97200 2024-09-18T01:03:59.344 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:03:59.350 INFO:teuthology.orchestra.run.smithi164.stdout:4923 read 0x3f7b800 thru 0x3f8b4b8 (0xfcb9 bytes) 2024-09-18T01:03:59.357 INFO:teuthology.orchestra.run.smithi164.stdout:4926 read 0x97c9e00 thru 0x97cc34a (0x254b bytes) 2024-09-18T01:03:59.358 INFO:teuthology.orchestra.run.smithi164.stdout:4927 trunc from 0xaa97200 to 0xae0b600 2024-09-18T01:03:59.372 INFO:teuthology.orchestra.run.smithi164.stdout:4928 trunc from 0xae0b600 to 0x7eed200 2024-09-18T01:03:59.390 INFO:teuthology.orchestra.run.smithi164.stdout:4931 trunc from 0x7eed200 to 0x400a400 2024-09-18T01:03:59.399 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:59.398+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:59.430 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:59.430 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:59.434 INFO:teuthology.orchestra.run.smithi164.stdout:4932 write 0xc3e6a00 thru 0xc3f3aa8 (0xd0a9 bytes) 2024-09-18T01:03:59.437 INFO:teuthology.orchestra.run.smithi164.stdout:4934 trunc from 0xc3f3c00 to 0xe6d1800 2024-09-18T01:03:59.440 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:03:59.466 INFO:teuthology.orchestra.run.smithi164.stdout:4936 read 0xd626200 thru 0xd629aef (0x38f0 bytes) 2024-09-18T01:03:59.466 INFO:teuthology.orchestra.run.smithi164.stdout:4939 write 0x56a9600 thru 0x56acbb7 (0x35b8 bytes) 2024-09-18T01:03:59.471 INFO:teuthology.orchestra.run.smithi164.stdout:4940 trunc from 0xe6d1800 to 0x9e46600 2024-09-18T01:03:59.499 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:59.498+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:59.500 INFO:teuthology.orchestra.run.smithi164.stdout:4941 read 0xdc4e00 thru 0xdc9d1e (0x4f1f bytes) 2024-09-18T01:03:59.500 INFO:teuthology.orchestra.run.smithi164.stdout:4944 punch from 0x8d7a200 to 0x8d89800, (0xf600 bytes) 2024-09-18T01:03:59.500 INFO:teuthology.orchestra.run.smithi164.stdout:4945 read 0x7b37e00 thru 0x7b38b04 (0xd05 bytes) 2024-09-18T01:03:59.507 INFO:teuthology.orchestra.run.smithi164.stdout:4946 trunc from 0x9e46600 to 0x47e8600 2024-09-18T01:03:59.535 INFO:teuthology.orchestra.run.smithi164.stdout:4948 trunc from 0x47e8600 to 0x6c44e00 2024-09-18T01:03:59.544 INFO:teuthology.orchestra.run.smithi164.stdout:4950 read 0x313b800 thru 0x31401ba (0x49bb bytes) 2024-09-18T01:03:59.545 INFO:teuthology.orchestra.run.smithi164.stdout:4951 read 0x4e4ca00 thru 0x4e53939 (0x6f3a bytes) 2024-09-18T01:03:59.545 INFO:teuthology.orchestra.run.smithi164.stdout:4952 read 0x6385600 thru 0x6392ee8 (0xd8e9 bytes) 2024-09-18T01:03:59.545 INFO:teuthology.orchestra.run.smithi164.stdout:4953 punch from 0x3c2d600 to 0x3c34a00, (0x7400 bytes) 2024-09-18T01:03:59.546 INFO:teuthology.orchestra.run.smithi164.stdout:4956 read 0x1a80400 thru 0x1a8b1c0 (0xadc1 bytes) 2024-09-18T01:03:59.546 INFO:teuthology.orchestra.run.smithi164.stdout:4957 read 0x2278200 thru 0x227b682 (0x3483 bytes) 2024-09-18T01:03:59.546 INFO:teuthology.orchestra.run.smithi164.stdout:4958 write 0x41e9c00 thru 0x41ecc3c (0x303d bytes) 2024-09-18T01:03:59.546 INFO:teuthology.orchestra.run.smithi164.stdout:4961 read 0x1878400 thru 0x1882d47 (0xa948 bytes) 2024-09-18T01:03:59.573 INFO:teuthology.orchestra.run.smithi164.stdout:4962 write 0xd15b800 thru 0xd1633e6 (0x7be7 bytes) 2024-09-18T01:03:59.573 INFO:teuthology.orchestra.run.smithi164.stdout:4963 write 0x2ac0800 thru 0x2ac545a (0x4c5b bytes) 2024-09-18T01:03:59.573 INFO:teuthology.orchestra.run.smithi164.stdout:4966 write 0xc931600 thru 0xc932835 (0x1236 bytes) 2024-09-18T01:03:59.574 INFO:teuthology.orchestra.run.smithi164.stdout:4972 write 0x60a7a00 thru 0x60a7d20 (0x321 bytes) 2024-09-18T01:03:59.574 INFO:teuthology.orchestra.run.smithi164.stdout:4973 read 0x4e18c00 thru 0x4e19208 (0x609 bytes) 2024-09-18T01:03:59.574 INFO:teuthology.orchestra.run.smithi164.stdout:4976 read 0xb2acc00 thru 0xb2b7c08 (0xb009 bytes) 2024-09-18T01:03:59.574 INFO:teuthology.orchestra.run.smithi164.stdout:4980 write 0x143e200 thru 0x1443568 (0x5369 bytes) 2024-09-18T01:03:59.574 INFO:teuthology.orchestra.run.smithi164.stdout:4981 punch from 0x9d01800 to 0x9d01a00, (0x200 bytes) 2024-09-18T01:03:59.575 INFO:teuthology.orchestra.run.smithi164.stdout:4982 write 0x2c59000 thru 0x2c5a62b (0x162c bytes) 2024-09-18T01:03:59.575 INFO:teuthology.orchestra.run.smithi164.stdout:4983 write 0x9668e00 thru 0x96782ee (0xf4ef bytes) 2024-09-18T01:03:59.575 INFO:teuthology.orchestra.run.smithi164.stdout:4984 write 0x2cb2a00 thru 0x2cbaaab (0x80ac bytes) 2024-09-18T01:03:59.575 INFO:teuthology.orchestra.run.smithi164.stdout:4985 punch from 0x4abb600 to 0x4aca000, (0xea00 bytes) 2024-09-18T01:03:59.580 INFO:teuthology.orchestra.run.smithi164.stdout:4986 trunc from 0xd163400 to 0x8ed3000 2024-09-18T01:03:59.601 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:59.601 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:59.601 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:03:59.600+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:03:59.611 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:03:59.619 INFO:teuthology.orchestra.run.smithi164.stdout:4987 read 0x26f8400 thru 0x26fdcd7 (0x58d8 bytes) 2024-09-18T01:03:59.625 INFO:teuthology.orchestra.run.smithi164.stdout:4988 punch from 0x1c8ea00 to 0x1c92400, (0x3a00 bytes) 2024-09-18T01:03:59.627 INFO:teuthology.orchestra.run.smithi164.stdout:4989 write 0x4a76600 thru 0x4a7fe88 (0x9889 bytes) 2024-09-18T01:03:59.628 INFO:teuthology.orchestra.run.smithi164.stdout:4990 write 0x80e6800 thru 0x80f626f (0xfa70 bytes) 2024-09-18T01:03:59.628 INFO:teuthology.orchestra.run.smithi164.stdout:4991 read 0x5755e00 thru 0x575c517 (0x6718 bytes) 2024-09-18T01:03:59.628 INFO:teuthology.orchestra.run.smithi164.stdout:4992 write 0x298cc00 thru 0x2995e0e (0x920f bytes) 2024-09-18T01:03:59.628 INFO:teuthology.orchestra.run.smithi164.stdout:4993 punch from 0x114c000 to 0x1151a00, (0x5a00 bytes) 2024-09-18T01:03:59.629 INFO:teuthology.orchestra.run.smithi164.stdout:4996 write 0xc16c00 thru 0xc16ccf (0xd0 bytes) 2024-09-18T01:03:59.629 INFO:teuthology.orchestra.run.smithi164.stdout:4997 read 0x41f5c00 thru 0x41f6b88 (0xf89 bytes) 2024-09-18T01:03:59.629 INFO:teuthology.orchestra.run.smithi164.stdout:4998 read 0x88c6c00 thru 0x88cc6a5 (0x5aa6 bytes) 2024-09-18T01:03:59.656 INFO:teuthology.orchestra.run.smithi164.stdout:5001 write 0xa3e4400 thru 0xa3e5a34 (0x1635 bytes) 2024-09-18T01:03:59.666 INFO:teuthology.orchestra.run.smithi164.stdout:5002 trunc from 0xa3e5c00 to 0x1d77a00 2024-09-18T01:03:59.702 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:03:59.701+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:03:59.716 INFO:teuthology.orchestra.run.smithi164.stdout:5003 write 0x231ec00 thru 0x232c34b (0xd74c bytes) 2024-09-18T01:03:59.720 INFO:teuthology.orchestra.run.smithi164.stdout:5004 trunc from 0x232c400 to 0x53dd600 2024-09-18T01:03:59.726 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:59.726 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:59.736 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:03:59.759 INFO:teuthology.orchestra.run.smithi164.stdout:5006 write 0xc97fa00 thru 0xc987b9a (0x819b bytes) 2024-09-18T01:03:59.760 INFO:teuthology.orchestra.run.smithi164.stdout:5007 read 0x7792e00 thru 0x77a1149 (0xe34a bytes) 2024-09-18T01:03:59.760 INFO:teuthology.orchestra.run.smithi164.stdout:5008 write 0xbea6200 thru 0xbeaaa16 (0x4817 bytes) 2024-09-18T01:03:59.760 INFO:teuthology.orchestra.run.smithi164.stdout:5009 punch from 0x21a9600 to 0x21b0c00, (0x7600 bytes) 2024-09-18T01:03:59.791 INFO:teuthology.orchestra.run.smithi164.stdout:5010 write 0xca23000 thru 0xca2e09c (0xb09d bytes) 2024-09-18T01:03:59.791 INFO:teuthology.orchestra.run.smithi164.stdout:5011 write 0x777e600 thru 0x7786032 (0x7a33 bytes) 2024-09-18T01:03:59.791 INFO:teuthology.orchestra.run.smithi164.stdout:5015 read 0x1c4aa00 thru 0x1c4b08a (0x68b bytes) 2024-09-18T01:03:59.791 INFO:teuthology.orchestra.run.smithi164.stdout:5017 write 0xa86f400 thru 0xa87c32d (0xcf2e bytes) 2024-09-18T01:03:59.791 INFO:teuthology.orchestra.run.smithi164.stdout:5018 trunc from 0xca2e200 to 0xd0c5e00 2024-09-18T01:03:59.813 INFO:teuthology.orchestra.run.smithi164.stdout:5020 read 0x7639800 thru 0x763d40a (0x3c0b bytes) 2024-09-18T01:03:59.817 INFO:teuthology.orchestra.run.smithi164.stdout:5022 trunc from 0xd0c5e00 to 0xaab0e00 2024-09-18T01:03:59.821 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:03:59.820+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:03:59.836 INFO:teuthology.orchestra.run.smithi164.stdout:5023 write 0x1eee200 thru 0x1efdca4 (0xfaa5 bytes) 2024-09-18T01:03:59.848 INFO:teuthology.orchestra.run.smithi164.stdout:5025 trunc from 0xaab0e00 to 0x144ea00 2024-09-18T01:03:59.884 INFO:teuthology.orchestra.run.smithi164.stdout:5026 write 0x2c9e00 thru 0x2ce7dd (0x49de bytes) 2024-09-18T01:03:59.885 INFO:teuthology.orchestra.run.smithi164.stdout:5029 punch from 0xb27400 to 0xb31200, (0x9e00 bytes) 2024-09-18T01:03:59.894 INFO:teuthology.orchestra.run.smithi164.stdout:5030 trunc from 0x144ea00 to 0x8754c00 2024-09-18T01:03:59.898 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:03:59.898 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:03:59.903 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:03:59.902+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:03:59.903 INFO:teuthology.orchestra.run.smithi164.stdout:5031 read 0x15dca00 thru 0x15e26f8 (0x5cf9 bytes) 2024-09-18T01:03:59.907 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:03:59.912 INFO:teuthology.orchestra.run.smithi164.stdout:5032 trunc from 0x8754c00 to 0x148d400 2024-09-18T01:03:59.926 INFO:teuthology.orchestra.run.smithi164.stdout:5034 read 0xe6cc00 thru 0xe6ceac (0x2ad bytes) 2024-09-18T01:03:59.942 INFO:teuthology.orchestra.run.smithi164.stdout:5037 write 0x65af000 thru 0x65b0037 (0x1038 bytes) 2024-09-18T01:03:59.943 INFO:teuthology.orchestra.run.smithi164.stdout:5039 read 0x2f68200 thru 0x2f712ba (0x90bb bytes) 2024-09-18T01:03:59.943 INFO:teuthology.orchestra.run.smithi164.stdout:5040 read 0x170e00 thru 0x173817 (0x2a18 bytes) 2024-09-18T01:03:59.946 INFO:teuthology.orchestra.run.smithi164.stdout:5043 trunc from 0x65b0200 to 0x46dce00 2024-09-18T01:03:59.967 INFO:teuthology.orchestra.run.smithi164.stdout:5044 read 0xf67e00 thru 0xf69569 (0x176a bytes) 2024-09-18T01:03:59.972 INFO:teuthology.orchestra.run.smithi164.stdout:5045 trunc from 0x46dce00 to 0x1489400 2024-09-18T01:03:59.989 INFO:teuthology.orchestra.run.smithi164.stdout:5046 read 0x2a0600 thru 0x2a3c7b (0x367c bytes) 2024-09-18T01:03:59.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:00.004 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:00.003+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:00.005 INFO:teuthology.orchestra.run.smithi164.stdout:5048 trunc from 0x1489400 to 0xba94a00 2024-09-18T01:04:00.030 INFO:teuthology.orchestra.run.smithi164.stdout:5052 trunc from 0xba94a00 to 0x364200 2024-09-18T01:04:00.057 INFO:teuthology.orchestra.run.smithi164.stdout:5055 read 0x19f200 thru 0x1a96e4 (0xa4e5 bytes) 2024-09-18T01:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout:5056 read 0x9da00 thru 0xabb0d (0xe10e bytes) 2024-09-18T01:04:00.060 INFO:teuthology.orchestra.run.smithi164.stdout:5057 read 0x29d600 thru 0x2a9bab (0xc5ac bytes) 2024-09-18T01:04:00.060 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:00.061 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:00.069 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:00.089 INFO:teuthology.orchestra.run.smithi164.stdout:5058 write 0xece0000 thru 0xece8fc2 (0x8fc3 bytes) 2024-09-18T01:04:00.090 INFO:teuthology.orchestra.run.smithi164.stdout:5059 punch from 0x36e4400 to 0x36ef200, (0xae00 bytes) 2024-09-18T01:04:00.091 INFO:teuthology.orchestra.run.smithi164.stdout:5060 write 0x419a00 thru 0x425efe (0xc4ff bytes) 2024-09-18T01:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout:5065 write 0xc091e00 thru 0xc093a46 (0x1c47 bytes) 2024-09-18T01:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout:5066 read 0x9452a00 thru 0x945c737 (0x9d38 bytes) 2024-09-18T01:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout:5067 write 0x2512400 thru 0x251dfd1 (0xbbd2 bytes) 2024-09-18T01:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout:5068 read 0x494a600 thru 0x494d0a4 (0x2aa5 bytes) 2024-09-18T01:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout:5071 read 0x6381000 thru 0x6388103 (0x7104 bytes) 2024-09-18T01:04:00.092 INFO:teuthology.orchestra.run.smithi164.stdout:5072 read 0x8570c00 thru 0x857793d (0x6d3e bytes) 2024-09-18T01:04:00.098 INFO:teuthology.orchestra.run.smithi164.stdout:5074 trunc from 0xece9000 to 0x8f77000 2024-09-18T01:04:00.111 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:00.110+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:00.144 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:00.144 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:00.152 INFO:teuthology.orchestra.run.smithi164.stdout:5076 punch from 0x1b5ae00 to 0x1b66c00, (0xbe00 bytes) 2024-09-18T01:04:00.154 INFO:teuthology.orchestra.run.smithi164.stdout:5079 punch from 0x6747800 to 0x6751200, (0x9a00 bytes) 2024-09-18T01:04:00.156 INFO:teuthology.orchestra.run.smithi164.stdout:5080 trunc from 0x8f77000 to 0x8261e00 2024-09-18T01:04:00.165 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:00.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:00.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:00.185 INFO:teuthology.orchestra.run.smithi164.stdout:5082 write 0x8214000 thru 0x82191f4 (0x51f5 bytes) 2024-09-18T01:04:00.185 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:00.189 INFO:teuthology.orchestra.run.smithi164.stdout:5084 read 0x5daea00 thru 0x5db62ca (0x78cb bytes) 2024-09-18T01:04:00.190 INFO:teuthology.orchestra.run.smithi164.stdout:5085 read 0x54c5a00 thru 0x54c9338 (0x3939 bytes) 2024-09-18T01:04:00.190 INFO:teuthology.orchestra.run.smithi164.stdout:5087 read 0x2400200 thru 0x240577e (0x557f bytes) 2024-09-18T01:04:00.190 INFO:teuthology.orchestra.run.smithi164.stdout:5089 write 0x583fc00 thru 0x584998d (0x9d8e bytes) 2024-09-18T01:04:00.196 INFO:teuthology.orchestra.run.smithi164.stdout:5093 trunc from 0x8261e00 to 0xd3f0200 2024-09-18T01:04:00.200 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:00.201 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:00.205 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:00.204+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:00.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:00.220 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.930074748000003, 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.228 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.229 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:00.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:00.231 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:00.232 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.930050631, 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.233 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.234 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:00.235 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.236 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.929971391000002, 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:00.237 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.238 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:00.239 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.240 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.929935172, 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.241 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.242 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.243 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:00.244 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.929906559000003, 2024-09-18T01:04:00.245 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.246 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.247 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.248 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.249 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.929885693999999, 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.250 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.251 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.252 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:00.252 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:00.252 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.252 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.252 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.252 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:00.252 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:00.253 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.253 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.253 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.253 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:00.253 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:00.253 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.253 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.254 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.254 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:00.254 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:00.254 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.254 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.254 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.254 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:00.255 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:00.255 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.255 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.255 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.255 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:00.255 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:00.256 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.256 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.256 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.256 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.256 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.256 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.257 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:00.257 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.929267977000002, 2024-09-18T01:04:00.257 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:00.257 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.257 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.257 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.258 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.258 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.258 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.258 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:00.258 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.258 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.258 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.259 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.259 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:00.259 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.259 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.259 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.259 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.260 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:00.260 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.260 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.260 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.260 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.260 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:00.261 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:00.261 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.261 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.261 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.261 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:00.261 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:00.262 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.262 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.262 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.262 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:00.262 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:00.262 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.262 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.263 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.263 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:00.263 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:00.263 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.263 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.263 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:00.264 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.265 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.265 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.265 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.265 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.265 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.265 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:00.265 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.92923871, 2024-09-18T01:04:00.266 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:00.266 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.266 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.266 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.266 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.267 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.267 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.267 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:00.267 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.267 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.267 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.268 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.268 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:00.268 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.268 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.268 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.269 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:00.269 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.269 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.269 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.269 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:00.270 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:00.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.270 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.270 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.270 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:00.271 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:00.271 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.271 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.271 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.271 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:00.271 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:00.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.272 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.272 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.272 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:00.272 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:00.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.272 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.273 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.273 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:00.273 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:00.273 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.273 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.273 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.929172418999997, 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.274 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.275 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:00.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.277 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:00.278 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.929051651999998, 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:00.279 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.280 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:00.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.282 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928833206, 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:00.284 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.285 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:00.286 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928811844999998, 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.287 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:00.289 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.290 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:00.291 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928657905999998, 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.292 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:00.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:00.294 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928638188000001, 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.296 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.297 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:00.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.299 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:00.300 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928617877999997, 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.301 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.303 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:00.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928597992, 2024-09-18T01:04:00.305 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:00.306 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.307 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.308 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.309 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928578033000001, 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.310 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:00.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.312 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.313 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928558035000002, 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.314 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.315 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:00.316 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.317 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928538367999998, 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.318 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.319 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.320 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:00.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 53.928518466, 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:00.322 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:00.324 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.325 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.326 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:00.327 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:00.329 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:00.329 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:00.327+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:00.329 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:00.429 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:00.428+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:00.472 INFO:teuthology.orchestra.run.smithi164.stdout:5094 punch from 0x9763a00 to 0x976fc00, (0xc200 bytes) 2024-09-18T01:04:00.472 INFO:teuthology.orchestra.run.smithi164.stdout:5095 write 0xa3fe600 thru 0xa402918 (0x4319 bytes) 2024-09-18T01:04:00.472 INFO:teuthology.orchestra.run.smithi164.stdout:5100 read 0x7f73c00 thru 0x7f77999 (0x3d9a bytes) 2024-09-18T01:04:00.472 INFO:teuthology.orchestra.run.smithi164.stdout:5101 punch from 0xb23c000 to 0xb24b200, (0xf200 bytes) 2024-09-18T01:04:00.473 INFO:teuthology.orchestra.run.smithi164.stdout:5102 punch from 0x1e5a00 to 0x1eca00, (0x7000 bytes) 2024-09-18T01:04:00.473 INFO:teuthology.orchestra.run.smithi164.stdout:5105 write 0x444b200 thru 0x4450ad6 (0x58d7 bytes) 2024-09-18T01:04:00.473 INFO:teuthology.orchestra.run.smithi164.stdout:5106 read 0x96db400 thru 0x96e7735 (0xc336 bytes) 2024-09-18T01:04:00.473 INFO:teuthology.orchestra.run.smithi164.stdout:5108 read 0xcc5400 thru 0xccbd1d (0x691e bytes) 2024-09-18T01:04:00.473 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:00.473 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:00.474 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:00.474 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:00.474 INFO:teuthology.orchestra.run.smithi164.stdout:5110 read 0x8316400 thru 0x831e13e (0x7d3f bytes) 2024-09-18T01:04:00.474 INFO:teuthology.orchestra.run.smithi164.stdout:5112 read 0x3377600 thru 0x337e84a (0x724b bytes) 2024-09-18T01:04:00.474 INFO:teuthology.orchestra.run.smithi164.stdout:5114 write 0x4656200 thru 0x4662cb0 (0xcab1 bytes) 2024-09-18T01:04:00.475 INFO:teuthology.orchestra.run.smithi164.stdout:5118 write 0xbd5b600 thru 0xbd63e1a (0x881b bytes) 2024-09-18T01:04:00.475 INFO:teuthology.orchestra.run.smithi164.stdout:5120 write 0x1394400 thru 0x13a3010 (0xec11 bytes) 2024-09-18T01:04:00.476 INFO:teuthology.orchestra.run.smithi164.stdout:5121 trunc from 0xd3f0200 to 0xb379000 2024-09-18T01:04:00.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:00.517 INFO:teuthology.orchestra.run.smithi164.stdout:5122 write 0x79dd200 thru 0x79e6e20 (0x9c21 bytes) 2024-09-18T01:04:00.522 INFO:teuthology.orchestra.run.smithi164.stdout:5123 read 0x46c7600 thru 0x46cd657 (0x6058 bytes) 2024-09-18T01:04:00.523 INFO:teuthology.orchestra.run.smithi164.stdout:5124 trunc from 0xb379000 to 0xaf98e00 2024-09-18T01:04:00.540 INFO:teuthology.orchestra.run.smithi164.stdout:5125 read 0x29ba600 thru 0x29c4027 (0x9a28 bytes) 2024-09-18T01:04:00.545 INFO:teuthology.orchestra.run.smithi164.stdout:5126 write 0x716c800 thru 0x7173335 (0x6b36 bytes) 2024-09-18T01:04:00.545 INFO:teuthology.orchestra.run.smithi164.stdout:5127 punch from 0x5cfe600 to 0x5cff800, (0x1200 bytes) 2024-09-18T01:04:00.560 INFO:teuthology.orchestra.run.smithi164.stdout:5128 write 0xe642000 thru 0xe645c4c (0x3c4d bytes) 2024-09-18T01:04:00.560 INFO:teuthology.orchestra.run.smithi164.stdout:5130 punch from 0x1d19800 to 0x1d1be00, (0x2600 bytes) 2024-09-18T01:04:00.565 INFO:teuthology.orchestra.run.smithi164.stdout:5131 punch from 0x2e77000 to 0x2e7b200, (0x4200 bytes) 2024-09-18T01:04:00.565 INFO:teuthology.orchestra.run.smithi164.stdout:5132 read 0xa193e00 thru 0xa19cd98 (0x8f99 bytes) 2024-09-18T01:04:00.565 INFO:teuthology.orchestra.run.smithi164.stdout:5133 read 0x2822e00 thru 0x282ed72 (0xbf73 bytes) 2024-09-18T01:04:00.565 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:00.564+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:00.566 INFO:teuthology.orchestra.run.smithi164.stdout:5135 write 0x8c75800 thru 0x8c7c31a (0x6b1b bytes) 2024-09-18T01:04:00.566 INFO:teuthology.orchestra.run.smithi164.stdout:5136 read 0xaefca00 thru 0xaefd1df (0x7e0 bytes) 2024-09-18T01:04:00.567 INFO:teuthology.orchestra.run.smithi164.stdout:5140 punch from 0x4a0f000 to 0x4a1bc00, (0xcc00 bytes) 2024-09-18T01:04:00.567 INFO:teuthology.orchestra.run.smithi164.stdout:5141 punch from 0x8435200 to 0x843c400, (0x7200 bytes) 2024-09-18T01:04:00.567 INFO:teuthology.orchestra.run.smithi164.stdout:5145 read 0x84f0c00 thru 0x84f54c2 (0x48c3 bytes) 2024-09-18T01:04:00.567 INFO:teuthology.orchestra.run.smithi164.stdout:5146 write 0x4235e00 thru 0x4239383 (0x3584 bytes) 2024-09-18T01:04:00.567 INFO:teuthology.orchestra.run.smithi164.stdout:5147 punch from 0xe327c00 to 0xe330400, (0x8800 bytes) 2024-09-18T01:04:00.567 INFO:teuthology.orchestra.run.smithi164.stdout:5148 clone 29 order 20 su 524288 sc 14 2024-09-18T01:04:00.631 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:00.630+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:00.633 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:00.633 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:00.633 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:00.633 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:00.634 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:00.642 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:00.732 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:00.731+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:00.787 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:00.787 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:00.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:00.787 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:00.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.787 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.410968616000005, 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.788 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:00.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:00.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:00.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:00.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:00.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:00.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:00.797 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 87.412957311, 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.798 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:00.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.805 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.405280359000002, 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:00.806 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:00.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.307422408999997, 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:00.813 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.814 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:00.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.820 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.451341396999993, 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.821 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.823 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:00.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.450972101000005, 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.827 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:00.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:00.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.833 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.449233255999999, 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.834 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:00.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.449161365999998, 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:00.838 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:00.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:00.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.842 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.361295355999999, 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.843 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:00.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:00.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.355104096999995, 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.849 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:00.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.852 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:00.853 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.353574412, 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.854 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:00.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:00.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.858 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.353194723000001, 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:00.860 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:00.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:00.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.866 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.337696320000006, 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:00.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:00.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:00.872 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.337489560999998, 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.873 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:00.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:00.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.337071804000004, 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.879 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:00.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.883 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.336862565999994, 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.884 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:00.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:00.888 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.336798631999997, 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:00.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.336050427000004, 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.893 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:00.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.896 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.897 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.077111622000004, 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.898 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.899 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.900 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.901 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:00.902 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.903 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 73.076248004000007, 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:00.904 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.905 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:00.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:00.907 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:00.909 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:00.910 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:00.912 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:00.910+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:00.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:01.012 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:01.011+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:01.023 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:01.023 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:01.024 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:01.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:01.113 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:01.112+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:01.179 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:01.179 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:01.179 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:01.180 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:01.180 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:01.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:01.214 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:01.213+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.976393008000002, 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.976372499, 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.976352693999999, 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.976318054000004, 2024-09-18T01:04:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.976296552999997, 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.976273376000002, 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972284543999997, 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972264991000003, 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972244486999998, 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972223622999998, 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972203723, 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972179635000003, 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972148314999998, 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972103709000002, 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972072918000002, 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.371 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972049433000002, 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.972020378000003, 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:01.376 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:01.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:01.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.971999619000002, 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:01.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:01.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.714058871999999, 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.386 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:01.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.713975163000001, 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.397 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:01.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:04:01.398 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:01.397+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:01.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:01.499 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:01.497+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:01.552 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5285.0:11664 3.2 3:402597a0:::rbd_data.11037ce5c9b3.0000000000000008:head [read 8192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:04:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:01.551287+0000", 2024-09-18T01:04:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000118071, 2024-09-18T01:04:01.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014398300000000001, 2024-09-18T01:04:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5285", 2024-09-18T01:04:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/52906593", 2024-09-18T01:04:01.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11664 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551287+0000", 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551287+0000", 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551288+0000", 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1099999999999995e-07 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551295+0000", 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1339999999999998e-06 2024-09-18T01:04:01.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551306+0000", 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0808999999999999e-05 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551317+0000", 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1187e-05 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551372+0000", 2024-09-18T01:04:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5433999999999998e-05 2024-09-18T01:04:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551436+0000", 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3516999999999995e-05 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5285.0:11666 3.2 3.b2a6bc82 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:01.551345+0000", 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0504999999999998e-05, 2024-09-18T01:04:01.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016101599999999999, 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5285", 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/52906593", 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11666 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551345+0000", 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551345+0000", 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551346+0000", 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5499999999999997e-07 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551349+0000", 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7900000000000001e-06 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551359+0000", 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7909999999999999e-06 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551365+0000", 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0419999999999999e-06 2024-09-18T01:04:01.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5285.0:11670 3.7 3:e6e8f8bb:::rbd_data.11037ce5c9b3.000000000000000e:head [read 0~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:01.551374+0000", 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.1399000000000002e-05, 2024-09-18T01:04:01.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019127900000000001, 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5285", 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/52906593", 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11670 2024-09-18T01:04:01.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551374+0000", 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551374+0000", 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551374+0000", 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551377+0000", 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5919999999999999e-06 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551382+0000", 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5910000000000003e-06 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551387+0000", 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.656e-06 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:01.551515+0000", 2024-09-18T01:04:01.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000128221 2024-09-18T01:04:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:01.568 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 3 2024-09-18T01:04:01.568 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:01.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:01.611 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:01.610+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:01.700 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:01.699+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:01.722 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:01.722 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:01.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:01.801 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:01.799+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:01.822 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.603974117000007, 2024-09-18T01:04:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.599348214000003, 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.599270164999993, 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.542474122000002, 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.542448258, 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.542425778000002, 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.542404742000002, 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.542377348000002, 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.542353042999999, 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538953560000003, 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:04:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538929000000003, 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:04:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:04:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:04:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:04:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538887483000003, 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:04:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:04:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538866980999998, 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538846515000003, 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538805175, 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538785136999998, 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538742806000002, 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538721813999999, 2024-09-18T01:04:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.538670844000002, 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.306530456000001, 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:01.998 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:02.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:02.001 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:01.998+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:02.002 INFO:teuthology.orchestra.run.smithi164.stdout:5149 trunc from 0xe645e00 to 0x9990a00 2024-09-18T01:04:02.002 INFO:teuthology.orchestra.run.smithi164.stdout:5151 punch from 0x1b5ba00 to 0x1b60600, (0x4c00 bytes) 2024-09-18T01:04:02.002 INFO:teuthology.orchestra.run.smithi164.stdout:5156 write 0x334e600 thru 0x3350682 (0x2083 bytes) 2024-09-18T01:04:02.002 INFO:teuthology.orchestra.run.smithi164.stdout:5157 trunc from 0x9990a00 to 0x2654800 2024-09-18T01:04:02.002 INFO:teuthology.orchestra.run.smithi164.stdout:5159 trunc from 0x2654800 to 0x4a16a00 2024-09-18T01:04:02.002 INFO:teuthology.orchestra.run.smithi164.stdout:5160 read 0x1d12e00 thru 0x1d21652 (0xe853 bytes) 2024-09-18T01:04:02.002 INFO:teuthology.orchestra.run.smithi164.stdout:5162 punch from 0x2384200 to 0x238e400, (0xa200 bytes) 2024-09-18T01:04:02.003 INFO:teuthology.orchestra.run.smithi164.stdout:5163 trunc from 0x4a16a00 to 0x8966a00 2024-09-18T01:04:02.003 INFO:teuthology.orchestra.run.smithi164.stdout:5164 read 0x2952e00 thru 0x2960f21 (0xe122 bytes) 2024-09-18T01:04:02.003 INFO:teuthology.orchestra.run.smithi164.stdout:5165 write 0x15a9400 thru 0x15ab019 (0x1c1a bytes) 2024-09-18T01:04:02.003 INFO:teuthology.orchestra.run.smithi164.stdout:5166 write 0x916b200 thru 0x916c966 (0x1767 bytes) 2024-09-18T01:04:02.003 INFO:teuthology.orchestra.run.smithi164.stdout:5167 punch from 0x26d8e00 to 0x26d9600, (0x800 bytes) 2024-09-18T01:04:02.003 INFO:teuthology.orchestra.run.smithi164.stdout:5170 clone 30 order 21 su 8192 sc 12 2024-09-18T01:04:02.101 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:02.100+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:02.147 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:02.147 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:02.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:02.201 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:02.200+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:02.263 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:02.263 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:02.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:02.315 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:02.314+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:02.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:02.349 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:04:02.388 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.652006642000003, 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:02.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:02.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.651986684000001, 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.651967743, 2024-09-18T01:04:02.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:02.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.651946234, 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:02.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:02.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.651920506000003, 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:02.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:02.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:02.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643188647000002, 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:04:02.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:04:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:04:02.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643165932999999, 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:02.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643146803, 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643127956999997, 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:02.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:02.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:02.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:02.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643109078999998, 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:02.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:02.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:02.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643087213999998, 2024-09-18T01:04:02.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:02.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643063579, 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:02.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:02.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:02.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643029929000001, 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:02.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.643002074000002, 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:02.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:02.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:02.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.642982250999999, 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:02.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.642961817, 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:02.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.642939736999999, 2024-09-18T01:04:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:02.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:02.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:02.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:02.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.642917414999999, 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:02.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.642895019000001, 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:02.506 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:02.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.512 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.642874599999999, 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:02.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:02.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:02.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:02.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:02.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:02.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:02.520 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:02.521 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone29 from 0x916ca00 (overlap 0x2654800) to 0x227f400 2024-09-18T01:04:02.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:02.523 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:02.523 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:02.524 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:02.524 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:02.520+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:02.529 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:02.529 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:02.547 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:02.547 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:02.566 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:02.566 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:02.622 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:02.621+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:02.629 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:02.629 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:02.637 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:02.724 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:02.723+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:02.755 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-18T01:04:02.785 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:02.785 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:02.793 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:02.837 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:02.836+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:02.926 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:02.924+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:02.929 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:02.929 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:02.940 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:03.027 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:03.026+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:03.118 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:03.119 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:03.128 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:03.127+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:03.130 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:03.229 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:03.227+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:03.311 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:03.311 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:03.322 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:03.329 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:03.328+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:03.431 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:03.430+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:03.489 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:03.489 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:03.498 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:03.532 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:03.531+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:03.633 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:03.632+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:03.640 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:03.640 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:03.650 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:03.680 INFO:teuthology.orchestra.run.smithi164.stdout:5172 read 0x3c3cc00 thru 0x3c3ed54 (0x2155 bytes) 2024-09-18T01:04:03.680 INFO:teuthology.orchestra.run.smithi164.stdout:5173 punch from 0x5190200 to 0x5194600, (0x4400 bytes) 2024-09-18T01:04:03.695 INFO:teuthology.orchestra.run.smithi164.stdout:5174 read 0x2acba00 thru 0x2ad6b7e (0xb17f bytes) 2024-09-18T01:04:03.696 INFO:teuthology.orchestra.run.smithi164.stdout:5175 read 0x1fe400 thru 0x20a928 (0xc529 bytes) 2024-09-18T01:04:03.701 INFO:teuthology.orchestra.run.smithi164.stdout:5178 trunc from 0x916ca00 to 0x5011400 2024-09-18T01:04:03.729 INFO:teuthology.orchestra.run.smithi164.stdout:5179 read 0x1e25800 thru 0x1e2806e (0x286f bytes) 2024-09-18T01:04:03.731 INFO:teuthology.orchestra.run.smithi164.stdout:5181 punch from 0x58d400 to 0x58fe00, (0x2a00 bytes) 2024-09-18T01:04:03.734 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:03.733+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:03.763 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:03.763 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:03.767 INFO:teuthology.orchestra.run.smithi164.stdout:5182 write 0xa7c9800 thru 0xa7ca0a9 (0x8aa bytes) 2024-09-18T01:04:03.772 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:03.790 INFO:teuthology.orchestra.run.smithi164.stdout:5184 write 0xc4fb600 thru 0xc508f85 (0xd986 bytes) 2024-09-18T01:04:03.792 INFO:teuthology.orchestra.run.smithi164.stdout:5185 trunc from 0xc509000 to 0xb566600 2024-09-18T01:04:03.825 INFO:teuthology.orchestra.run.smithi164.stdout:5187 write 0x39f3400 thru 0x39f3b8f (0x790 bytes) 2024-09-18T01:04:03.826 INFO:teuthology.orchestra.run.smithi164.stdout:5188 read 0x5261200 thru 0x526d914 (0xc715 bytes) 2024-09-18T01:04:03.826 INFO:teuthology.orchestra.run.smithi164.stdout:5189 write 0x261e400 thru 0x2624b84 (0x6785 bytes) 2024-09-18T01:04:03.835 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:03.834+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:03.873 INFO:teuthology.orchestra.run.smithi164.stdout:5190 write 0xdb9b000 thru 0xdba171d (0x671e bytes) 2024-09-18T01:04:03.879 INFO:teuthology.orchestra.run.smithi164.stdout:5192 trunc from 0xdba1800 to 0x8580c00 2024-09-18T01:04:03.890 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:03.890 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:03.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:03.910 INFO:teuthology.orchestra.run.smithi164.stdout:5193 write 0x2b86e00 thru 0x2b89c5f (0x2e60 bytes) 2024-09-18T01:04:03.910 INFO:teuthology.orchestra.run.smithi164.stdout:5195 punch from 0x4cbd800 to 0x4cc1c00, (0x4400 bytes) 2024-09-18T01:04:03.912 INFO:teuthology.orchestra.run.smithi164.stdout:5197 punch from 0x11cbe00 to 0x11d0400, (0x4600 bytes) 2024-09-18T01:04:03.936 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:03.935+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:03.980 INFO:teuthology.orchestra.run.smithi164.stdout:5198 write 0x9b6b600 thru 0x9b6e006 (0x2a07 bytes) 2024-09-18T01:04:03.991 INFO:teuthology.orchestra.run.smithi164.stdout:5201 trunc from 0x9b6e200 to 0x5bba00 2024-09-18T01:04:04.037 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:04.036+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:04.052 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:04.052 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:04.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:04.089 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:04:04.089 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:04:04.089 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-18T01:04:04.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-18T01:04:04.105 INFO:teuthology.orchestra.run.smithi164.stdout:5202 write 0x72bea00 thru 0x72c56d5 (0x6cd6 bytes) 2024-09-18T01:04:04.110 INFO:teuthology.orchestra.run.smithi164.stdout:5203 trunc from 0x72c5800 to 0x45dc000 2024-09-18T01:04:04.140 INFO:teuthology.orchestra.run.smithi164.stdout:5204 read 0x26c8000 thru 0x26d4eb7 (0xceb8 bytes) 2024-09-18T01:04:04.140 INFO:teuthology.orchestra.run.smithi164.stdout:5205 punch from 0x358e000 to 0x3591e00, (0x3e00 bytes) 2024-09-18T01:04:04.140 INFO:teuthology.orchestra.run.smithi164.stdout:5206 punch from 0x3485800 to 0x3493a00, (0xe200 bytes) 2024-09-18T01:04:04.140 INFO:teuthology.orchestra.run.smithi164.stdout:5207 read 0x4205a00 thru 0x4211d17 (0xc318 bytes) 2024-09-18T01:04:04.140 INFO:teuthology.orchestra.run.smithi164.stdout:5210 read 0x7ca800 thru 0x7d8b83 (0xe384 bytes) 2024-09-18T01:04:04.141 INFO:teuthology.orchestra.run.smithi164.stdout:5213 write 0x2c66400 thru 0x2c70adb (0xa6dc bytes) 2024-09-18T01:04:04.141 INFO:teuthology.orchestra.run.smithi164.stdout:5214 punch from 0x3d89e00 to 0x3d8f200, (0x5400 bytes) 2024-09-18T01:04:04.141 INFO:teuthology.orchestra.run.smithi164.stdout:5217 read 0x2dce600 thru 0x2dd89e7 (0xa3e8 bytes) 2024-09-18T01:04:04.142 INFO:teuthology.orchestra.run.smithi164.stdout:5218 punch from 0x1e93a00 to 0x1e9f000, (0xb600 bytes) 2024-09-18T01:04:04.150 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:04.149+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:04.188 INFO:teuthology.orchestra.run.smithi164.stdout:5220 write 0x91f4200 thru 0x91f6acf (0x28d0 bytes) 2024-09-18T01:04:04.188 INFO:teuthology.orchestra.run.smithi164.stdout:5223 write 0x80d1200 thru 0x80de8e2 (0xd6e3 bytes) 2024-09-18T01:04:04.189 INFO:teuthology.orchestra.run.smithi164.stdout:5226 trunc from 0x91f6c00 to 0x8325000 2024-09-18T01:04:04.237 INFO:teuthology.orchestra.run.smithi164.stdout:5227 write 0x59dae00 thru 0x59e68a3 (0xbaa4 bytes) 2024-09-18T01:04:04.237 INFO:teuthology.orchestra.run.smithi164.stdout:5228 punch from 0x7853c00 to 0x7859000, (0x5400 bytes) 2024-09-18T01:04:04.238 INFO:teuthology.orchestra.run.smithi164.stdout:5230 read 0x1f4200 thru 0x200e4b (0xcc4c bytes) 2024-09-18T01:04:04.239 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:04.238+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:04.240 INFO:teuthology.orchestra.run.smithi164.stdout:5231 write 0x624ea00 thru 0x625b433 (0xca34 bytes) 2024-09-18T01:04:04.240 INFO:teuthology.orchestra.run.smithi164.stdout:5234 write 0x4028200 thru 0x4037c3b (0xfa3c bytes) 2024-09-18T01:04:04.240 INFO:teuthology.orchestra.run.smithi164.stdout:5237 write 0x484c000 thru 0x484c897 (0x898 bytes) 2024-09-18T01:04:04.240 INFO:teuthology.orchestra.run.smithi164.stdout:5242 write 0x62afe00 thru 0x62b8570 (0x8771 bytes) 2024-09-18T01:04:04.244 INFO:teuthology.orchestra.run.smithi164.stdout:5244 trunc from 0x8325000 to 0x4d91400 2024-09-18T01:04:04.273 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:04.273 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:04.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-18T01:04:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-18T01:04:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-18T01:04:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-18T01:04:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-18T01:04:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-18T01:04:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-18T01:04:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:04:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-18T01:04:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-18T01:04:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-18T01:04:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-18T01:04:04.309 INFO:teuthology.orchestra.run.smithi164.stdout:5246 punch from 0x7d4c00 to 0x7d5000, (0x400 bytes) 2024-09-18T01:04:04.309 INFO:teuthology.orchestra.run.smithi164.stdout:5248 punch from 0xc2fa00 to 0xc3e200, (0xe800 bytes) 2024-09-18T01:04:04.309 INFO:teuthology.orchestra.run.smithi164.stdout:5249 write 0x1958e00 thru 0x19643b1 (0xb5b2 bytes) 2024-09-18T01:04:04.310 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:04:04.311 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:04:04.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-18T01:04:04.338 INFO:teuthology.orchestra.run.smithi164.stdout:5250 write 0xbe28000 thru 0xbe37ccf (0xfcd0 bytes) 2024-09-18T01:04:04.338 INFO:teuthology.orchestra.run.smithi164.stdout:5251 write 0x96e6200 thru 0x96e9515 (0x3316 bytes) 2024-09-18T01:04:04.339 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:04.338+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:04.346 INFO:teuthology.orchestra.run.smithi164.stdout:5253 trunc from 0xbe37e00 to 0x4fb1400 2024-09-18T01:04:04.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:04:04.395 INFO:teuthology.orchestra.run.smithi164.stdout:5254 read 0x37acc00 thru 0x37b4f2b (0x832c bytes) 2024-09-18T01:04:04.418 INFO:teuthology.orchestra.run.smithi164.stdout:5256 write 0x74b4a00 thru 0x74c1075 (0xc676 bytes) 2024-09-18T01:04:04.418 INFO:teuthology.orchestra.run.smithi164.stdout:5258 read 0x27e4000 thru 0x27ec0b1 (0x80b2 bytes) 2024-09-18T01:04:04.418 INFO:teuthology.orchestra.run.smithi164.stdout:5259 write 0x666de00 thru 0x667133f (0x3540 bytes) 2024-09-18T01:04:04.418 INFO:teuthology.orchestra.run.smithi164.stdout:5262 read 0x1b55200 thru 0x1b650de (0xfedf bytes) 2024-09-18T01:04:04.420 INFO:teuthology.orchestra.run.smithi164.stdout:5264 read 0x195e600 thru 0x1969629 (0xb02a bytes) 2024-09-18T01:04:04.421 INFO:teuthology.orchestra.run.smithi164.stdout:5266 read 0x57f2600 thru 0x58018d9 (0xf2da bytes) 2024-09-18T01:04:04.422 INFO:teuthology.orchestra.run.smithi164.stdout:5269 read 0x4133400 thru 0x4137a0c (0x460d bytes) 2024-09-18T01:04:04.422 INFO:teuthology.orchestra.run.smithi164.stdout:5271 read 0x3bffc00 thru 0x3c0cf42 (0xd343 bytes) 2024-09-18T01:04:04.441 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:04.439+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:04.456 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:04.456 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:04.462 INFO:teuthology.orchestra.run.smithi164.stdout:5272 write 0x9468600 thru 0x9471a56 (0x9457 bytes) 2024-09-18T01:04:04.462 INFO:teuthology.orchestra.run.smithi164.stdout:5273 read 0x7c99e00 thru 0x7ca843b (0xe63c bytes) 2024-09-18T01:04:04.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:04.471 INFO:teuthology.orchestra.run.smithi164.stdout:5274 trunc from 0x9471c00 to 0x19f9c00 2024-09-18T01:04:04.533 INFO:teuthology.orchestra.run.smithi164.stdout:5276 clone 31 order 23 su 65536 sc 14 2024-09-18T01:04:04.540 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-18T01:04:04.540 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-18T01:04:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-18T01:04:04.542 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:04.541+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:04.544 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:04:04.544 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:04:04.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-18T01:04:04.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:04.599 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:04.600 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:04.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:04.650 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:04.649+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:04.714 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:04.714 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:04.733 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:04.733 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:04.744 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:04.743+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:04.750 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:04.750 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-18T01:04:04.751 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:04:04.752 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-18T01:04:04.752 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-18T01:04:04.752 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-18T01:04:04.752 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-18T01:04:04.754 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:04:04.754 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:04:04.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-18T01:04:04.764 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:04.764 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:04.764 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:04.764 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:04.764 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:04.768 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:04.768 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:04.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:04.787 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:04.787 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:04.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:04.845 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:04.844+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:04.899 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.620402655000007, 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.615776752000002, 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.615698703000007, 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558902660000001, 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558876796000007, 2024-09-18T01:04:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558854315999994, 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558833280000002, 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558805886000002, 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558781581000005, 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555382097999995, 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:04:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555357537999996, 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:04:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555316020999996, 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555295518999998, 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555275053000003, 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555233713000007, 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555213675000005, 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555171344000001, 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555150351999998, 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.555099381999995, 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.322958994, 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.004 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:05.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:04:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-18T01:04:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-18T01:04:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-18T01:04:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-18T01:04:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-18T01:04:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-18T01:04:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-18T01:04:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:04:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-18T01:04:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-18T01:04:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-18T01:04:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-18T01:04:05.007 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:04:05.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-18T01:04:05.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:05.009 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:05.005+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:05.107 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:05.106+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:05.157 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:05.158 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:05.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-09-18T01:04:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-09-18T01:04:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-09-18T01:04:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-09-18T01:04:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-09-18T01:04:05.184 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-09-18T01:04:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-09-18T01:04:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-09-18T01:04:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-09-18T01:04:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-09-18T01:04:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-09-18T01:04:05.185 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-09-18T01:04:05.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-18T01:04:05.187 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-18T01:04:05.229 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:05.228+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:05.291 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:05.291 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:05.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:05.341 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:05.340+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:05.409 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:05.408+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:05.412 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.722303676999999, 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.722283719000004, 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.722264778000003, 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.722243269000003, 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.722217540999999, 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713485681999998, 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713462968000002, 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713443838000003, 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713424992, 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713406114000001, 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713384249000001, 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713360614000003, 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713326963999997, 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713299108999998, 2024-09-18T01:04:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713279286000002, 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713258852000003, 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713236772000002, 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713214450000002, 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713192053999997, 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.713171635000002, 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:05.580 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:05.581 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone30 from 0x19f9c00 (overlap 0x5bba00) to 0x1b8400 2024-09-18T01:04:05.581 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:05.580+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:05.582 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:05.683 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:05.681+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:05.687 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:05.687 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:05.687 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:05.687 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:05.696 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:05.784 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:05.783+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:05.803 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:05.803 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:05.803 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:05.803 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:05.803 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:05.812 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:05.882 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-18T01:04:05.897 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:05.896+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:05.976 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:05.976 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:05.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:05.976 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:05.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.976 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.930358007999999, 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:05.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:04:05.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.930346857000004, 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:04:05.981 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:04:05.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.985 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.930335026999998, 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:04:05.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.990 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.930318344, 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.991 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.992 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:04:05.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:05.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:05.995 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.930307106999997, 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:05.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:04:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:04:05.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.928985001999997, 2024-09-18T01:04:06.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:06.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:04:06.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:04:06.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.928972287000001, 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.005 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:04:06.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.009 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.928927303000002, 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:04:06.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:04:06.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.928899678999997, 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.015 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:04:06.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.9253523, 2024-09-18T01:04:06.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:06.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.924992193000001, 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:04:06.024 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:06.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.924966079999997, 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.029 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:04:06.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:04:06.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.922786692000003, 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:04:06.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.922776896999999, 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.039 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:06.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:04:06.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.922766117999998, 2024-09-18T01:04:06.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:04:06.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.922746322000002, 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.048 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:04:06.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:04:06.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.920962801999998, 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:04:06.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.057 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.920935964000002, 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.058 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:04:06.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 63.920925416000003, 2024-09-18T01:04:06.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:06.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:04:06.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:04:06.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 28.448998540000002, 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.067 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:04:06.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:04:06.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:04:06.073 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.073 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.073 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.073 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.073 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.073 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:06.074 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:06.075 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:06.073+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:06.088 INFO:teuthology.orchestra.run.smithi164.stdout:5279 punch from 0x9c3600 to 0x9c4400, (0xe00 bytes) 2024-09-18T01:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout:5280 write 0xd1600 thru 0xdb131 (0x9b32 bytes) 2024-09-18T01:04:06.129 INFO:teuthology.orchestra.run.smithi164.stdout:5283 read 0xe05000 thru 0xe06da9 (0x1daa bytes) 2024-09-18T01:04:06.130 INFO:teuthology.orchestra.run.smithi164.stdout:5284 punch from 0x361e00 to 0x371800, (0xfa00 bytes) 2024-09-18T01:04:06.165 INFO:teuthology.orchestra.run.smithi164.stdout:5289 read 0x1111e00 thru 0x1114cad (0x2eae bytes) 2024-09-18T01:04:06.175 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:06.174+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:06.227 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:06.228 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:06.228 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:06.228 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:06.236 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:06.280 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:06.279+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:06.339 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:06.339 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:06.339 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:06.339 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:06.339 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:06.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:06.378 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:06.377+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:06.428 INFO:teuthology.orchestra.run.smithi164.stdout:5290 write 0x352ba00 thru 0x35363a8 (0xa9a9 bytes) 2024-09-18T01:04:06.433 INFO:teuthology.orchestra.run.smithi164.stdout:5291 trunc from 0x3536400 to 0x72eb800 2024-09-18T01:04:06.464 INFO:teuthology.orchestra.run.smithi164.stdout:5292 write 0x115600 thru 0x12025d (0xac5e bytes) 2024-09-18T01:04:06.465 INFO:teuthology.orchestra.run.smithi164.stdout:5295 read 0xc2ca00 thru 0xc3933a (0xc93b bytes) 2024-09-18T01:04:06.466 INFO:teuthology.orchestra.run.smithi164.stdout:5296 read 0x67e1800 thru 0x67e57cd (0x3fce bytes) 2024-09-18T01:04:06.466 INFO:teuthology.orchestra.run.smithi164.stdout:5297 write 0x3b9fa00 thru 0x3bab6b1 (0xbcb2 bytes) 2024-09-18T01:04:06.467 INFO:teuthology.orchestra.run.smithi164.stdout:5300 punch from 0x63d3800 to 0x63d3c00, (0x400 bytes) 2024-09-18T01:04:06.472 INFO:teuthology.orchestra.run.smithi164.stdout:5302 read 0x233f800 thru 0x234f40d (0xfc0e bytes) 2024-09-18T01:04:06.473 INFO:teuthology.orchestra.run.smithi164.stdout:5303 flatten 2024-09-18T01:04:06.479 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:06.478+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:06.506 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:06.506 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.208473871999999, 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:06.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:06.512 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.208449754999997, 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:06.516 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.208370514999999, 2024-09-18T01:04:06.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:06.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.521 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.208334295999997, 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.522 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:06.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:06.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.208305682999999, 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.533 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.208284818000003, 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.534 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:06.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207667100999998, 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:06.538 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:06.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207637834000003, 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:06.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:06.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207571543, 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.547 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:06.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207450776000002, 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:06.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207232329999997, 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.556 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:06.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:06.560 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207210969000002, 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:06.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.564 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207057030000001, 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.565 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:06.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207037311999997, 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.569 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:06.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.207017002000001, 2024-09-18T01:04:06.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:06.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:06.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.206997115999997, 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.578 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:06.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.206977156999997, 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.582 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:06.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.586 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.206957158999998, 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:06.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.206937492000002, 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.591 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:06.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.595 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 60.206917590000003, 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:06.596 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:06.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:06.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:06.600 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:06.601 INFO:teuthology.orchestra.run.smithi164.stdout:5304 write 0xbe7d600 thru 0xbe8d297 (0xfc98 bytes) 2024-09-18T01:04:06.602 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:06.602 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:06.600+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:06.603 INFO:teuthology.orchestra.run.smithi164.stdout:5305 read 0xa825400 thru 0xa834284 (0xee85 bytes) 2024-09-18T01:04:06.604 INFO:teuthology.orchestra.run.smithi164.stdout:5306 trunc from 0xbe8d400 to 0xb603800 2024-09-18T01:04:06.633 INFO:teuthology.orchestra.run.smithi164.stdout:5307 write 0xea7ae00 thru 0xea86dbe (0xbfbf bytes) 2024-09-18T01:04:06.640 INFO:teuthology.orchestra.run.smithi164.stdout:5309 write 0x26bc600 thru 0x26ca1f2 (0xdbf3 bytes) 2024-09-18T01:04:06.641 INFO:teuthology.orchestra.run.smithi164.stdout:5310 write 0x6be7000 thru 0x6bec1ce (0x51cf bytes) 2024-09-18T01:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout:5314 write 0x9562c00 thru 0x956fcea (0xd0eb bytes) 2024-09-18T01:04:06.642 INFO:teuthology.orchestra.run.smithi164.stdout:5315 write 0x2f2e600 thru 0x2f3a017 (0xba18 bytes) 2024-09-18T01:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout:5318 write 0xb180000 thru 0xb18c0da (0xc0db bytes) 2024-09-18T01:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout:5320 read 0x5e19600 thru 0x5e277fc (0xe1fd bytes) 2024-09-18T01:04:06.643 INFO:teuthology.orchestra.run.smithi164.stdout:5321 read 0x3bf7a00 thru 0x3bfde32 (0x6433 bytes) 2024-09-18T01:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout:5323 read 0x3a39000 thru 0x3a3b31d (0x231e bytes) 2024-09-18T01:04:06.644 INFO:teuthology.orchestra.run.smithi164.stdout:5325 write 0x6f59a00 thru 0x6f67bed (0xe1ee bytes) 2024-09-18T01:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout:5326 read 0xb791400 thru 0xb796ece (0x5acf bytes) 2024-09-18T01:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout:5327 write 0x2ac3200 thru 0x2ad1208 (0xe009 bytes) 2024-09-18T01:04:06.645 INFO:teuthology.orchestra.run.smithi164.stdout:5329 read 0xe18800 thru 0xe2224b (0x9a4c bytes) 2024-09-18T01:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout:5330 read 0x906a800 thru 0x90728f2 (0x80f3 bytes) 2024-09-18T01:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout:5331 write 0x753da00 thru 0x75455fa (0x7bfb bytes) 2024-09-18T01:04:06.646 INFO:teuthology.orchestra.run.smithi164.stdout:5332 write 0x2c9fe00 thru 0x2ca3f49 (0x414a bytes) 2024-09-18T01:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout:5333 write 0xdfe9c00 thru 0xdfefd51 (0x6152 bytes) 2024-09-18T01:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout:5335 write 0xa3b7800 thru 0xa3bb208 (0x3a09 bytes) 2024-09-18T01:04:06.647 INFO:teuthology.orchestra.run.smithi164.stdout:5337 write 0x102e200 thru 0x103a9e7 (0xc7e8 bytes) 2024-09-18T01:04:06.656 INFO:teuthology.orchestra.run.smithi164.stdout:5338 trunc from 0xea86e00 to 0x6bcac00 2024-09-18T01:04:06.698 INFO:teuthology.orchestra.run.smithi164.stdout:5339 write 0x1c38200 thru 0x1c3f0ef (0x6ef0 bytes) 2024-09-18T01:04:06.701 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:06.700+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:06.707 INFO:teuthology.orchestra.run.smithi164.stdout:5340 trunc from 0x6bcac00 to 0xce29000 2024-09-18T01:04:06.718 INFO:teuthology.orchestra.run.smithi164.stdout:5342 write 0x4e59600 thru 0x4e67ecf (0xe8d0 bytes) 2024-09-18T01:04:06.719 INFO:teuthology.orchestra.run.smithi164.stdout:5343 write 0xb30cc00 thru 0xb31a8f1 (0xdcf2 bytes) 2024-09-18T01:04:06.720 INFO:teuthology.orchestra.run.smithi164.stdout:5344 trunc from 0xce29000 to 0xda4ca00 2024-09-18T01:04:06.734 INFO:teuthology.orchestra.run.smithi164.stdout:5345 write 0x47d9600 thru 0x47df53b (0x5f3c bytes) 2024-09-18T01:04:06.749 INFO:teuthology.orchestra.run.smithi164.stdout:5348 trunc from 0xda4ca00 to 0x36fa800 2024-09-18T01:04:06.762 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:06.762 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:06.762 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:06.762 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:06.771 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:06.780 INFO:teuthology.orchestra.run.smithi164.stdout:5351 trunc from 0x36fa800 to 0xa639e00 2024-09-18T01:04:06.789 INFO:teuthology.orchestra.run.smithi164.stdout:5352 punch from 0x40f600 to 0x411400, (0x1e00 bytes) 2024-09-18T01:04:06.790 INFO:teuthology.orchestra.run.smithi164.stdout:5353 write 0x81cea00 thru 0x81d630f (0x7910 bytes) 2024-09-18T01:04:06.799 INFO:teuthology.orchestra.run.smithi164.stdout:5355 trunc from 0xa639e00 to 0x3340200 2024-09-18T01:04:06.812 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:06.811+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:06.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:06.829 INFO:teuthology.orchestra.run.smithi164.stdout:5358 read 0x12b0200 thru 0x12bc2d1 (0xc0d2 bytes) 2024-09-18T01:04:06.841 INFO:teuthology.orchestra.run.smithi164.stdout:5361 write 0x49d9800 thru 0x49daac5 (0x12c6 bytes) 2024-09-18T01:04:06.842 INFO:teuthology.orchestra.run.smithi164.stdout:5364 punch from 0xa8ba00 to 0xa91800, (0x5e00 bytes) 2024-09-18T01:04:06.858 INFO:teuthology.orchestra.run.smithi164.stdout:5365 write 0x5002600 thru 0x500e741 (0xc142 bytes) 2024-09-18T01:04:06.859 INFO:teuthology.orchestra.run.smithi164.stdout:5366 read 0x2dd1600 thru 0x2ddfc29 (0xe62a bytes) 2024-09-18T01:04:06.860 INFO:teuthology.orchestra.run.smithi164.stdout:5368 write 0x43f1800 thru 0x43fa304 (0x8b05 bytes) 2024-09-18T01:04:06.860 INFO:teuthology.orchestra.run.smithi164.stdout:5372 read 0x3549000 thru 0x3556ac9 (0xdaca bytes) 2024-09-18T01:04:06.864 INFO:teuthology.orchestra.run.smithi164.stdout:5375 trunc from 0x500e800 to 0x7aec000 2024-09-18T01:04:06.876 INFO:teuthology.orchestra.run.smithi164.stdout:5376 read 0x469ee00 thru 0x46a93a1 (0xa5a2 bytes) 2024-09-18T01:04:06.877 INFO:teuthology.orchestra.run.smithi164.stdout:5379 read 0x4817e00 thru 0x482563b (0xd83c bytes) 2024-09-18T01:04:06.882 INFO:teuthology.orchestra.run.smithi164.stdout:5380 trunc from 0x7aec000 to 0xb49a000 2024-09-18T01:04:06.892 INFO:teuthology.orchestra.run.smithi164.stdout:5385 read 0x2269400 thru 0x226e01e (0x4c1f bytes) 2024-09-18T01:04:06.894 INFO:teuthology.orchestra.run.smithi164.stdout:5386 punch from 0xab2f000 to 0xab3be00, (0xce00 bytes) 2024-09-18T01:04:06.894 INFO:teuthology.orchestra.run.smithi164.stdout:5387 trunc from 0xb49a000 to 0xafa3c00 2024-09-18T01:04:06.903 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:06.902+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:06.925 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:06.925 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:06.925 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:06.926 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:06.926 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:06.929 INFO:teuthology.orchestra.run.smithi164.stdout:5392 write 0xd52cc00 thru 0xd532ab7 (0x5eb8 bytes) 2024-09-18T01:04:06.936 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:06.941 INFO:teuthology.orchestra.run.smithi164.stdout:5394 trunc from 0xd532c00 to 0x3f04400 2024-09-18T01:04:06.949 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:06.949 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:06.965 INFO:teuthology.orchestra.run.smithi164.stdout:5395 read 0x31ece00 thru 0x31f8e99 (0xc09a bytes) 2024-09-18T01:04:06.968 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:06.968 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:06.985 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:06.986 INFO:teuthology.orchestra.run.smithi164.stdout:5396 write 0xb6f6600 thru 0xb706378 (0xfd79 bytes) 2024-09-18T01:04:06.987 INFO:teuthology.orchestra.run.smithi164.stdout:5400 punch from 0xd3ac00 to 0xd3e200, (0x3600 bytes) 2024-09-18T01:04:06.988 INFO:teuthology.orchestra.run.smithi164.stdout:5403 punch from 0x168a400 to 0x1690800, (0x6400 bytes) 2024-09-18T01:04:06.988 INFO:teuthology.orchestra.run.smithi164.stdout:5406 write 0x99c8800 thru 0x99d5392 (0xcb93 bytes) 2024-09-18T01:04:06.988 INFO:teuthology.orchestra.run.smithi164.stdout:5407 write 0x3ff8000 thru 0x40008a6 (0x88a7 bytes) 2024-09-18T01:04:07.004 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:07.003+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:07.006 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:07.006 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:07.009 INFO:teuthology.orchestra.run.smithi164.stdout:5409 write 0xe749800 thru 0xe74ee35 (0x5636 bytes) 2024-09-18T01:04:07.010 INFO:teuthology.orchestra.run.smithi164.stdout:5411 write 0x9ee200 thru 0x9f7c84 (0x9a85 bytes) 2024-09-18T01:04:07.011 INFO:teuthology.orchestra.run.smithi164.stdout:5412 read 0xdc54400 thru 0xdc54def (0x9f0 bytes) 2024-09-18T01:04:07.022 INFO:teuthology.orchestra.run.smithi164.stdout:5413 trunc from 0xe74f000 to 0x5113c00 2024-09-18T01:04:07.024 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:07.024 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:07.044 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:07.044 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:07.070 INFO:teuthology.orchestra.run.smithi164.stdout:5414 write 0x9227000 thru 0x9230660 (0x9661 bytes) 2024-09-18T01:04:07.071 INFO:teuthology.orchestra.run.smithi164.stdout:5415 write 0x3061800 thru 0x3069c34 (0x8435 bytes) 2024-09-18T01:04:07.072 INFO:teuthology.orchestra.run.smithi164.stdout:5418 read 0x15c9a00 thru 0x15d673b (0xcd3c bytes) 2024-09-18T01:04:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:5419 read 0x71c9200 thru 0x71cc326 (0x3127 bytes) 2024-09-18T01:04:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:5420 write 0x75cb400 thru 0x75ce677 (0x3278 bytes) 2024-09-18T01:04:07.073 INFO:teuthology.orchestra.run.smithi164.stdout:5421 write 0x57f3200 thru 0x57f59f9 (0x27fa bytes) 2024-09-18T01:04:07.083 INFO:teuthology.orchestra.run.smithi164.stdout:5423 trunc from 0x9230800 to 0x148aa00 2024-09-18T01:04:07.090 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:07.090 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:07.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:07.105 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:07.104+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:07.124 INFO:teuthology.orchestra.run.smithi164.stdout:5424 trunc from 0x148aa00 to 0x5d59800 2024-09-18T01:04:07.151 INFO:teuthology.orchestra.run.smithi164.stdout:5425 write 0xd7eee00 thru 0xd7f159d (0x279e bytes) 2024-09-18T01:04:07.152 INFO:teuthology.orchestra.run.smithi164.stdout:5427 read 0x2cee800 thru 0x2cf0ffd (0x27fe bytes) 2024-09-18T01:04:07.164 INFO:teuthology.orchestra.run.smithi164.stdout:5428 trunc from 0xd7f1600 to 0x3567600 2024-09-18T01:04:07.201 INFO:teuthology.orchestra.run.smithi164.stdout:5434 punch from 0x1d53e00 to 0x1d5f200, (0xb400 bytes) 2024-09-18T01:04:07.203 INFO:teuthology.orchestra.run.smithi164.stdout:5435 read 0xdf5000 thru 0xdf9708 (0x4709 bytes) 2024-09-18T01:04:07.203 INFO:teuthology.orchestra.run.smithi164.stdout:5436 read 0x2978a00 thru 0x2985950 (0xcf51 bytes) 2024-09-18T01:04:07.205 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:07.204+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:07.207 INFO:teuthology.orchestra.run.smithi164.stdout:5440 trunc from 0x3567600 to 0x5d85800 2024-09-18T01:04:07.215 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:07.215 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:07.216 INFO:teuthology.orchestra.run.smithi164.stdout:5441 read 0x3844a00 thru 0x38548e1 (0xfee2 bytes) 2024-09-18T01:04:07.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:07.234 INFO:teuthology.orchestra.run.smithi164.stdout:5444 write 0xc97a200 thru 0xc982754 (0x8555 bytes) 2024-09-18T01:04:07.234 INFO:teuthology.orchestra.run.smithi164.stdout:5445 punch from 0x424ea00 to 0x4253200, (0x4800 bytes) 2024-09-18T01:04:07.235 INFO:teuthology.orchestra.run.smithi164.stdout:5447 read 0x8282800 thru 0x82859f3 (0x31f4 bytes) 2024-09-18T01:04:07.235 INFO:teuthology.orchestra.run.smithi164.stdout:5450 read 0x6f23a00 thru 0x6f2c0cb (0x86cc bytes) 2024-09-18T01:04:07.236 INFO:teuthology.orchestra.run.smithi164.stdout:5451 write 0xb049a00 thru 0xb055a5e (0xc05f bytes) 2024-09-18T01:04:07.236 INFO:teuthology.orchestra.run.smithi164.stdout:5453 read 0x279be00 thru 0x279e0b3 (0x22b4 bytes) 2024-09-18T01:04:07.237 INFO:teuthology.orchestra.run.smithi164.stdout:5454 read 0x7493200 thru 0x7496118 (0x2f19 bytes) 2024-09-18T01:04:07.238 INFO:teuthology.orchestra.run.smithi164.stdout:5458 write 0x2ba5000 thru 0x2bada98 (0x8a99 bytes) 2024-09-18T01:04:07.238 INFO:teuthology.orchestra.run.smithi164.stdout:5459 read 0xa3aa800 thru 0xa3b96b5 (0xeeb6 bytes) 2024-09-18T01:04:07.242 INFO:teuthology.orchestra.run.smithi164.stdout:5460 trunc from 0xc982800 to 0xa1b6a00 2024-09-18T01:04:07.256 INFO:teuthology.orchestra.run.smithi164.stdout:5461 read 0x8e80400 thru 0x8e88847 (0x8448 bytes) 2024-09-18T01:04:07.257 INFO:teuthology.orchestra.run.smithi164.stdout:5462 trunc from 0xa1b6a00 to 0xa3b5600 2024-09-18T01:04:07.276 INFO:teuthology.orchestra.run.smithi164.stdout:5466 write 0xb876400 thru 0xb879e30 (0x3a31 bytes) 2024-09-18T01:04:07.276 INFO:teuthology.orchestra.run.smithi164.stdout:5469 write 0x177c000 thru 0x1786166 (0xa167 bytes) 2024-09-18T01:04:07.291 INFO:teuthology.orchestra.run.smithi164.stdout:5470 write 0xc09d400 thru 0xc09fe78 (0x2a79 bytes) 2024-09-18T01:04:07.298 INFO:teuthology.orchestra.run.smithi164.stdout:5471 trunc from 0xc0a0000 to 0xbb9ae00 2024-09-18T01:04:07.308 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:07.307+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:07.314 INFO:teuthology.orchestra.run.smithi164.stdout:5476 write 0x56b8800 thru 0x56bad5d (0x255e bytes) 2024-09-18T01:04:07.332 INFO:teuthology.orchestra.run.smithi164.stdout:5477 write 0xbc6bc00 thru 0xbc73787 (0x7b88 bytes) 2024-09-18T01:04:07.348 INFO:teuthology.orchestra.run.smithi164.stdout:5478 write 0xdbb4000 thru 0xdbbb805 (0x7806 bytes) 2024-09-18T01:04:07.358 INFO:teuthology.orchestra.run.smithi164.stdout:5479 trunc from 0xdbbba00 to 0x649a600 2024-09-18T01:04:07.382 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:07.382 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:07.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:07.402 INFO:teuthology.orchestra.run.smithi164.stdout:5485 write 0xdf98c00 thru 0xdfa5d00 (0xd101 bytes) 2024-09-18T01:04:07.402 INFO:teuthology.orchestra.run.smithi164.stdout:5486 punch from 0xb7b3c00 to 0xb7b5000, (0x1400 bytes) 2024-09-18T01:04:07.403 INFO:teuthology.orchestra.run.smithi164.stdout:5489 read 0x4583a00 thru 0x458f47c (0xba7d bytes) 2024-09-18T01:04:07.404 INFO:teuthology.orchestra.run.smithi164.stdout:5490 read 0xacb1600 thru 0xacb2ab0 (0x14b1 bytes) 2024-09-18T01:04:07.417 INFO:teuthology.orchestra.run.smithi164.stdout:5491 trunc from 0xdfa5e00 to 0x283f400 2024-09-18T01:04:07.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:04:07.432 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:07.431+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:07.447 INFO:teuthology.orchestra.run.smithi164.stdout:5495 punch from 0x14ff800 to 0x1501000, (0x1800 bytes) 2024-09-18T01:04:07.465 INFO:teuthology.orchestra.run.smithi164.stdout:5496 write 0x8a01600 thru 0x8a0d1bb (0xbbbc bytes) 2024-09-18T01:04:07.465 INFO:teuthology.orchestra.run.smithi164.stdout:5499 read 0x696a800 thru 0x6972f26 (0x8727 bytes) 2024-09-18T01:04:07.481 INFO:teuthology.orchestra.run.smithi164.stdout:5501 write 0xc0f2a00 thru 0xc1003bf (0xd9c0 bytes) 2024-09-18T01:04:07.482 INFO:teuthology.orchestra.run.smithi164.stdout:5503 read 0x9358e00 thru 0x935a48a (0x168b bytes) 2024-09-18T01:04:07.483 INFO:teuthology.orchestra.run.smithi164.stdout:5504 write 0x336a400 thru 0x3378423 (0xe024 bytes) 2024-09-18T01:04:07.488 INFO:teuthology.orchestra.run.smithi164.stdout:5506 trunc from 0xc100400 to 0x7c41200 2024-09-18T01:04:07.508 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:07.507+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:07.509 INFO:teuthology.orchestra.run.smithi164.stdout:5508 write 0x72ac00 thru 0x734ebc (0xa2bd bytes) 2024-09-18T01:04:07.515 INFO:teuthology.orchestra.run.smithi164.stdout:5509 trunc from 0x7c41200 to 0xc107c00 2024-09-18T01:04:07.526 INFO:teuthology.orchestra.run.smithi164.stdout:5511 read 0x485c00 thru 0x486147 (0x548 bytes) 2024-09-18T01:04:07.527 INFO:teuthology.orchestra.run.smithi164.stdout:5513 punch from 0x2948a00 to 0x2958800, (0xfe00 bytes) 2024-09-18T01:04:07.528 INFO:teuthology.orchestra.run.smithi164.stdout:5514 write 0xa1c9c00 thru 0xa1d1f38 (0x8339 bytes) 2024-09-18T01:04:07.528 INFO:teuthology.orchestra.run.smithi164.stdout:5515 punch from 0x2458000 to 0x245f800, (0x7800 bytes) 2024-09-18T01:04:07.528 INFO:teuthology.orchestra.run.smithi164.stdout:5517 read 0xa87a000 thru 0xa883273 (0x9274 bytes) 2024-09-18T01:04:07.532 INFO:teuthology.orchestra.run.smithi164.stdout:5518 trunc from 0xc107c00 to 0xe394800 2024-09-18T01:04:07.550 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:07.551 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:07.552 INFO:teuthology.orchestra.run.smithi164.stdout:5519 write 0xe52c200 thru 0xe537c0f (0xba10 bytes) 2024-09-18T01:04:07.552 INFO:teuthology.orchestra.run.smithi164.stdout:5523 read 0xdd54c00 thru 0xdd55803 (0xc04 bytes) 2024-09-18T01:04:07.553 INFO:teuthology.orchestra.run.smithi164.stdout:5524 punch from 0xe081800 to 0xe08d000, (0xb800 bytes) 2024-09-18T01:04:07.559 INFO:teuthology.orchestra.run.smithi164.stdout:5527 trunc from 0xe537e00 to 0x96fee00 2024-09-18T01:04:07.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:07.586 INFO:teuthology.orchestra.run.smithi164.stdout:5528 write 0x446e200 thru 0x446e95b (0x75c bytes) 2024-09-18T01:04:07.598 INFO:teuthology.orchestra.run.smithi164.stdout:5529 write 0x9b8a000 thru 0x9b964d4 (0xc4d5 bytes) 2024-09-18T01:04:07.599 INFO:teuthology.orchestra.run.smithi164.stdout:5530 trunc from 0x9b96600 to 0xa4ef800 2024-09-18T01:04:07.609 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:07.608+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:07.611 INFO:teuthology.orchestra.run.smithi164.stdout:5531 read 0xfa4000 thru 0xfa98ca (0x58cb bytes) 2024-09-18T01:04:07.618 INFO:teuthology.orchestra.run.smithi164.stdout:5536 read 0x1e68200 thru 0x1e699df (0x17e0 bytes) 2024-09-18T01:04:07.621 INFO:teuthology.orchestra.run.smithi164.stdout:5538 trunc from 0xa4ef800 to 0x78f0400 2024-09-18T01:04:07.644 INFO:teuthology.orchestra.run.smithi164.stdout:5539 trunc from 0x78f0400 to 0xa9e5600 2024-09-18T01:04:07.663 INFO:teuthology.orchestra.run.smithi164.stdout:5540 trunc from 0xa9e5600 to 0x6ad1400 2024-09-18T01:04:07.688 INFO:teuthology.orchestra.run.smithi164.stdout:5541 read 0x2b5d400 thru 0x2b631e4 (0x5de5 bytes) 2024-09-18T01:04:07.688 INFO:teuthology.orchestra.run.smithi164.stdout:5543 trunc from 0x6ad1400 to 0x6636200 2024-09-18T01:04:07.699 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:07.699 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:07.702 INFO:teuthology.orchestra.run.smithi164.stdout:5545 punch from 0x623c200 to 0x623dc00, (0x1a00 bytes) 2024-09-18T01:04:07.704 INFO:teuthology.orchestra.run.smithi164.stdout:5546 write 0x5b9f000 thru 0x5b9f478 (0x479 bytes) 2024-09-18T01:04:07.704 INFO:teuthology.orchestra.run.smithi164.stdout:5547 read 0x3068e00 thru 0x3076ba9 (0xddaa bytes) 2024-09-18T01:04:07.705 INFO:teuthology.orchestra.run.smithi164.stdout:5548 read 0x618cc00 thru 0x6193bc3 (0x6fc4 bytes) 2024-09-18T01:04:07.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:07.710 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:07.709+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:07.719 INFO:teuthology.orchestra.run.smithi164.stdout:5549 write 0x9747e00 thru 0x975508f (0xd290 bytes) 2024-09-18T01:04:07.720 INFO:teuthology.orchestra.run.smithi164.stdout:5551 read 0x7b29800 thru 0x7b37523 (0xdd24 bytes) 2024-09-18T01:04:07.720 INFO:teuthology.orchestra.run.smithi164.stdout:5553 read 0x66e1000 thru 0x66f09fb (0xf9fc bytes) 2024-09-18T01:04:07.721 INFO:teuthology.orchestra.run.smithi164.stdout:5555 read 0xf4f200 thru 0xf56cfe (0x7aff bytes) 2024-09-18T01:04:07.724 INFO:teuthology.orchestra.run.smithi164.stdout:5557 read 0x34c5c00 thru 0x34c6105 (0x506 bytes) 2024-09-18T01:04:07.724 INFO:teuthology.orchestra.run.smithi164.stdout:5558 write 0x69c9200 thru 0x69cdf82 (0x4d83 bytes) 2024-09-18T01:04:07.725 INFO:teuthology.orchestra.run.smithi164.stdout:5559 write 0x1517000 thru 0x151bc8d (0x4c8e bytes) 2024-09-18T01:04:07.734 INFO:teuthology.orchestra.run.smithi164.stdout:5560 trunc from 0x9755200 to 0x20bd200 2024-09-18T01:04:07.786 INFO:teuthology.orchestra.run.smithi164.stdout:5562 write 0x51aa600 thru 0x51b8255 (0xdc56 bytes) 2024-09-18T01:04:07.787 INFO:teuthology.orchestra.run.smithi164.stdout:5563 read 0x36b0800 thru 0x36b59d7 (0x51d8 bytes) 2024-09-18T01:04:07.787 INFO:teuthology.orchestra.run.smithi164.stdout:5565 read 0x2f85a00 thru 0x2f8df2d (0x852e bytes) 2024-09-18T01:04:07.788 INFO:teuthology.orchestra.run.smithi164.stdout:5566 read 0x440f400 thru 0x4417c2e (0x882f bytes) 2024-09-18T01:04:07.792 INFO:teuthology.orchestra.run.smithi164.stdout:5568 trunc from 0x51b8400 to 0x1d1b000 2024-09-18T01:04:07.810 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:07.809+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:07.811 INFO:teuthology.orchestra.run.smithi164.stdout:5570 trunc from 0x1d1b000 to 0x476e600 2024-09-18T01:04:07.825 INFO:teuthology.orchestra.run.smithi164.stdout:5571 trunc from 0x476e600 to 0x5ebb600 2024-09-18T01:04:07.835 INFO:teuthology.orchestra.run.smithi164.stdout:5576 read 0x5d7a200 thru 0x5d7bcef (0x1af0 bytes) 2024-09-18T01:04:07.848 INFO:teuthology.orchestra.run.smithi164.stdout:5577 write 0x67ab600 thru 0x67ac1a2 (0xba3 bytes) 2024-09-18T01:04:07.860 INFO:teuthology.orchestra.run.smithi164.stdout:5578 write 0x6a25600 thru 0x6a26855 (0x1256 bytes) 2024-09-18T01:04:07.861 INFO:teuthology.orchestra.run.smithi164.stdout:5579 read 0x590a400 thru 0x59137cb (0x93cc bytes) 2024-09-18T01:04:07.861 INFO:teuthology.orchestra.run.smithi164.stdout:5580 write 0x83c800 thru 0x84a72f (0xdf30 bytes) 2024-09-18T01:04:07.862 INFO:teuthology.orchestra.run.smithi164.stdout:5581 read 0x4176800 thru 0x418018a (0x998b bytes) 2024-09-18T01:04:07.864 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:07.864 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:07.871 INFO:teuthology.orchestra.run.smithi164.stdout:5584 trunc from 0x6a26a00 to 0xe26e000 2024-09-18T01:04:07.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:07.882 INFO:teuthology.orchestra.run.smithi164.stdout:5588 write 0xab17c00 thru 0xab2002d (0x842e bytes) 2024-09-18T01:04:07.883 INFO:teuthology.orchestra.run.smithi164.stdout:5589 write 0x7a73a00 thru 0x7a81d50 (0xe351 bytes) 2024-09-18T01:04:07.883 INFO:teuthology.orchestra.run.smithi164.stdout:5591 write 0x45cde00 thru 0x45d61d9 (0x83da bytes) 2024-09-18T01:04:07.883 INFO:teuthology.orchestra.run.smithi164.stdout:5593 write 0x546c000 thru 0x54767e0 (0xa7e1 bytes) 2024-09-18T01:04:07.888 INFO:teuthology.orchestra.run.smithi164.stdout:5595 trunc from 0xe26e000 to 0xc8a0200 2024-09-18T01:04:07.909 INFO:teuthology.orchestra.run.smithi164.stdout:5596 write 0x7c46800 thru 0x7c4d878 (0x7079 bytes) 2024-09-18T01:04:07.911 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:07.910+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:07.917 INFO:teuthology.orchestra.run.smithi164.stdout:5597 punch from 0x90efc00 to 0x90f9800, (0x9c00 bytes) 2024-09-18T01:04:07.918 INFO:teuthology.orchestra.run.smithi164.stdout:5598 punch from 0x1bc5c00 to 0x1bd5600, (0xfa00 bytes) 2024-09-18T01:04:07.918 INFO:teuthology.orchestra.run.smithi164.stdout:5599 read 0x267dc00 thru 0x26869a5 (0x8da6 bytes) 2024-09-18T01:04:07.919 INFO:teuthology.orchestra.run.smithi164.stdout:5600 read 0x880a200 thru 0x88104da (0x62db bytes) 2024-09-18T01:04:07.933 INFO:teuthology.orchestra.run.smithi164.stdout:5601 trunc from 0xc8a0200 to 0x1114400 2024-09-18T01:04:07.965 INFO:teuthology.orchestra.run.smithi164.stdout:5604 read 0x2b600 thru 0x32769 (0x716a bytes) 2024-09-18T01:04:07.973 INFO:teuthology.orchestra.run.smithi164.stdout:5605 read 0xae2c00 thru 0xaf10d5 (0xe4d6 bytes) 2024-09-18T01:04:07.974 INFO:teuthology.orchestra.run.smithi164.stdout:5607 read 0x3c800 thru 0x46d26 (0xa527 bytes) 2024-09-18T01:04:07.974 INFO:teuthology.orchestra.run.smithi164.stdout:5608 read 0xcb000 thru 0xcdcf6 (0x2cf7 bytes) 2024-09-18T01:04:08.000 INFO:teuthology.orchestra.run.smithi164.stdout:5609 write 0x84a3400 thru 0x84a5b58 (0x2759 bytes) 2024-09-18T01:04:08.000 INFO:teuthology.orchestra.run.smithi164.stdout:5610 punch from 0x1a1bc00 to 0x1a20000, (0x4400 bytes) 2024-09-18T01:04:08.012 INFO:teuthology.orchestra.run.smithi164.stdout:5611 write 0xb6cf400 thru 0xb6d6868 (0x7469 bytes) 2024-09-18T01:04:08.013 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:08.012+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:08.029 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:08.029 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:08.037 INFO:teuthology.orchestra.run.smithi164.stdout:5612 write 0xd9b2e00 thru 0xd9b7c60 (0x4e61 bytes) 2024-09-18T01:04:08.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:08.039 INFO:teuthology.orchestra.run.smithi164.stdout:5613 write 0xca6d600 thru 0xca74fb0 (0x79b1 bytes) 2024-09-18T01:04:08.046 INFO:teuthology.orchestra.run.smithi164.stdout:5616 trunc from 0xd9b7e00 to 0x6d15800 2024-09-18T01:04:08.081 INFO:teuthology.orchestra.run.smithi164.stdout:5618 trunc from 0x6d15800 to 0xb250200 2024-09-18T01:04:08.091 INFO:teuthology.orchestra.run.smithi164.stdout:5619 punch from 0x32d2400 to 0x32df800, (0xd400 bytes) 2024-09-18T01:04:08.107 INFO:teuthology.orchestra.run.smithi164.stdout:5620 trunc from 0xb250200 to 0x1d82400 2024-09-18T01:04:08.123 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:08.122+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:08.129 INFO:teuthology.orchestra.run.smithi164.stdout:5622 write 0x180dc00 thru 0x1811f0b (0x430c bytes) 2024-09-18T01:04:08.144 INFO:teuthology.orchestra.run.smithi164.stdout:5624 trunc from 0x1d82400 to 0x8d16000 2024-09-18T01:04:08.156 INFO:teuthology.orchestra.run.smithi164.stdout:5625 write 0x411ae00 thru 0x4121b0e (0x6d0f bytes) 2024-09-18T01:04:08.158 INFO:teuthology.orchestra.run.smithi164.stdout:5626 write 0x592fe00 thru 0x593e847 (0xea48 bytes) 2024-09-18T01:04:08.158 INFO:teuthology.orchestra.run.smithi164.stdout:5630 write 0x2bd400 thru 0x2cd066 (0xfc67 bytes) 2024-09-18T01:04:08.159 INFO:teuthology.orchestra.run.smithi164.stdout:5631 read 0x634bc00 thru 0x634d8b8 (0x1cb9 bytes) 2024-09-18T01:04:08.159 INFO:teuthology.orchestra.run.smithi164.stdout:5635 write 0x55fa000 thru 0x55fbfaf (0x1fb0 bytes) 2024-09-18T01:04:08.166 INFO:teuthology.orchestra.run.smithi164.stdout:5636 trunc from 0x8d16000 to 0xe30de00 2024-09-18T01:04:08.183 INFO:teuthology.orchestra.run.smithi164.stdout:5637 read 0x3a22600 thru 0x3a268c1 (0x42c2 bytes) 2024-09-18T01:04:08.194 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:08.195 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:08.195 INFO:teuthology.orchestra.run.smithi164.stdout:5640 trunc from 0xe30de00 to 0x4bfd600 2024-09-18T01:04:08.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:08.212 INFO:teuthology.orchestra.run.smithi164.stdout:5644 read 0x4322400 thru 0x432fc96 (0xd897 bytes) 2024-09-18T01:04:08.213 INFO:teuthology.orchestra.run.smithi164.stdout:5645 write 0x8b0a00 thru 0x8b5ffe (0x55ff bytes) 2024-09-18T01:04:08.214 INFO:teuthology.orchestra.run.smithi164.stdout:5646 read 0x1039600 thru 0x103c4ec (0x2eed bytes) 2024-09-18T01:04:08.214 INFO:teuthology.orchestra.run.smithi164.stdout:5647 read 0x24d6e00 thru 0x24de511 (0x7712 bytes) 2024-09-18T01:04:08.215 INFO:teuthology.orchestra.run.smithi164.stdout:5648 write 0x12e4600 thru 0x12f3018 (0xea19 bytes) 2024-09-18T01:04:08.224 INFO:teuthology.orchestra.run.smithi164.stdout:5649 trunc from 0x4bfd600 to 0xbd77c00 2024-09-18T01:04:08.234 INFO:teuthology.orchestra.run.smithi164.stdout:5650 read 0xa1b3000 thru 0xa1c2731 (0xf732 bytes) 2024-09-18T01:04:08.237 INFO:teuthology.orchestra.run.smithi164.stdout:5651 trunc from 0xbd77c00 to 0xdae2200 2024-09-18T01:04:08.246 INFO:teuthology.orchestra.run.smithi164.stdout:5652 read 0xf25200 thru 0xf31b51 (0xc952 bytes) 2024-09-18T01:04:08.246 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:08.245+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:08.253 INFO:teuthology.orchestra.run.smithi164.stdout:5654 trunc from 0xdae2200 to 0xdbc6800 2024-09-18T01:04:08.262 INFO:teuthology.orchestra.run.smithi164.stdout:5655 read 0x2678e00 thru 0x268332f (0xa530 bytes) 2024-09-18T01:04:08.263 INFO:teuthology.orchestra.run.smithi164.stdout:5656 write 0x441da00 thru 0x442777e (0x9d7f bytes) 2024-09-18T01:04:08.264 INFO:teuthology.orchestra.run.smithi164.stdout:5659 read 0x37bf000 thru 0x37c9df4 (0xadf5 bytes) 2024-09-18T01:04:08.264 INFO:teuthology.orchestra.run.smithi164.stdout:5660 read 0x9ef3e00 thru 0x9f03563 (0xf764 bytes) 2024-09-18T01:04:08.264 INFO:teuthology.orchestra.run.smithi164.stdout:5661 read 0x87a800 thru 0x880921 (0x6122 bytes) 2024-09-18T01:04:08.265 INFO:teuthology.orchestra.run.smithi164.stdout:5662 read 0x2cec200 thru 0x2cfa4c5 (0xe2c6 bytes) 2024-09-18T01:04:08.265 INFO:teuthology.orchestra.run.smithi164.stdout:5663 read 0x275ce00 thru 0x2761df8 (0x4ff9 bytes) 2024-09-18T01:04:08.266 INFO:teuthology.orchestra.run.smithi164.stdout:5664 read 0x3302000 thru 0x330cf04 (0xaf05 bytes) 2024-09-18T01:04:08.267 INFO:teuthology.orchestra.run.smithi164.stdout:5668 read 0xc54a000 thru 0xc54bd32 (0x1d33 bytes) 2024-09-18T01:04:08.267 INFO:teuthology.orchestra.run.smithi164.stdout:5670 read 0x8a85400 thru 0x8a8d7b8 (0x83b9 bytes) 2024-09-18T01:04:08.267 INFO:teuthology.orchestra.run.smithi164.stdout:5673 read 0x82a4a00 thru 0x82aa640 (0x5c41 bytes) 2024-09-18T01:04:08.267 INFO:teuthology.orchestra.run.smithi164.stdout:5676 read 0x5787200 thru 0x57880d6 (0xed7 bytes) 2024-09-18T01:04:08.268 INFO:teuthology.orchestra.run.smithi164.stdout:5678 read 0x41a2000 thru 0x41a3d81 (0x1d82 bytes) 2024-09-18T01:04:08.268 INFO:teuthology.orchestra.run.smithi164.stdout:5680 punch from 0xa489a00 to 0xa491400, (0x7a00 bytes) 2024-09-18T01:04:08.268 INFO:teuthology.orchestra.run.smithi164.stdout:5681 read 0x5102200 thru 0x5103d6c (0x1b6d bytes) 2024-09-18T01:04:08.268 INFO:teuthology.orchestra.run.smithi164.stdout:5682 punch from 0xb4bc600 to 0xb4c7a00, (0xb400 bytes) 2024-09-18T01:04:08.269 INFO:teuthology.orchestra.run.smithi164.stdout:5688 read 0xcbe9c00 thru 0xcbea94d (0xd4e bytes) 2024-09-18T01:04:08.269 INFO:teuthology.orchestra.run.smithi164.stdout:5689 read 0x2896400 thru 0x289db1c (0x771d bytes) 2024-09-18T01:04:08.269 INFO:teuthology.orchestra.run.smithi164.stdout:5695 read 0x8154600 thru 0x8162313 (0xdd14 bytes) 2024-09-18T01:04:08.269 INFO:teuthology.orchestra.run.smithi164.stdout:5696 read 0x3dfbc00 thru 0x3e02b59 (0x6f5a bytes) 2024-09-18T01:04:08.271 INFO:teuthology.orchestra.run.smithi164.stdout:5697 write 0x1064800 thru 0x106b3f6 (0x6bf7 bytes) 2024-09-18T01:04:08.271 INFO:teuthology.orchestra.run.smithi164.stdout:5699 punch from 0xd71bc00 to 0xd71c000, (0x400 bytes) 2024-09-18T01:04:08.271 INFO:teuthology.orchestra.run.smithi164.stdout:5701 write 0x7eae200 thru 0x7eb0f8a (0x2d8b bytes) 2024-09-18T01:04:08.271 INFO:teuthology.orchestra.run.smithi164.stdout:5702 write 0x17a6000 thru 0x17abe16 (0x5e17 bytes) 2024-09-18T01:04:08.286 INFO:teuthology.orchestra.run.smithi164.stdout:5704 trunc from 0xdbc6800 to 0x19f2400 2024-09-18T01:04:08.314 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:08.313+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:08.330 INFO:teuthology.orchestra.run.smithi164.stdout:5705 trunc from 0x19f2400 to 0x9e0dc00 2024-09-18T01:04:08.349 INFO:teuthology.orchestra.run.smithi164.stdout:5706 write 0xa070000 thru 0xa0719af (0x19b0 bytes) 2024-09-18T01:04:08.349 INFO:teuthology.orchestra.run.smithi164.stdout:5710 punch from 0x6e96800 to 0x6e9a800, (0x4000 bytes) 2024-09-18T01:04:08.350 INFO:teuthology.orchestra.run.smithi164.stdout:5711 punch from 0x5dd8800 to 0x5ddcc00, (0x4400 bytes) 2024-09-18T01:04:08.350 INFO:teuthology.orchestra.run.smithi164.stdout:5714 write 0x358ea00 thru 0x3597c69 (0x926a bytes) 2024-09-18T01:04:08.353 INFO:teuthology.orchestra.run.smithi164.stdout:5715 write 0x34f8000 thru 0x3506016 (0xe017 bytes) 2024-09-18T01:04:08.353 INFO:teuthology.orchestra.run.smithi164.stdout:5717 write 0x6fe2a00 thru 0x6ff2408 (0xfa09 bytes) 2024-09-18T01:04:08.354 INFO:teuthology.orchestra.run.smithi164.stdout:5718 write 0x9fbf600 thru 0x9fc52e1 (0x5ce2 bytes) 2024-09-18T01:04:08.354 INFO:teuthology.orchestra.run.smithi164.stdout:5722 punch from 0x4b53a00 to 0x4b5b000, (0x7600 bytes) 2024-09-18T01:04:08.354 INFO:teuthology.orchestra.run.smithi164.stdout:5723 read 0x1766200 thru 0x1771d08 (0xbb09 bytes) 2024-09-18T01:04:08.355 INFO:teuthology.orchestra.run.smithi164.stdout:5724 punch from 0x797ac00 to 0x7988e00, (0xe200 bytes) 2024-09-18T01:04:08.363 INFO:teuthology.orchestra.run.smithi164.stdout:5725 trunc from 0xa071a00 to 0x4174e00 2024-09-18T01:04:08.366 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:08.366 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:08.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:08.393 INFO:teuthology.orchestra.run.smithi164.stdout:5726 write 0x2297c00 thru 0x22a3ead (0xc2ae bytes) 2024-09-18T01:04:08.402 INFO:teuthology.orchestra.run.smithi164.stdout:5727 trunc from 0x4174e00 to 0x5d71000 2024-09-18T01:04:08.415 INFO:teuthology.orchestra.run.smithi164.stdout:5730 read 0x2a45800 thru 0x2a507e3 (0xafe4 bytes) 2024-09-18T01:04:08.415 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:08.414+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:08.433 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:04:08.441 INFO:teuthology.orchestra.run.smithi164.stdout:5731 write 0xb607400 thru 0xb613265 (0xbe66 bytes) 2024-09-18T01:04:08.441 INFO:teuthology.orchestra.run.smithi164.stdout:5732 punch from 0x10b3c00 to 0x10bea00, (0xae00 bytes) 2024-09-18T01:04:08.442 INFO:teuthology.orchestra.run.smithi164.stdout:5733 write 0x674a600 thru 0x6759398 (0xed99 bytes) 2024-09-18T01:04:08.446 INFO:teuthology.orchestra.run.smithi164.stdout:5736 punch from 0x4a7be00 to 0x4a84000, (0x8200 bytes) 2024-09-18T01:04:08.447 INFO:teuthology.orchestra.run.smithi164.stdout:5737 write 0x3d55600 thru 0x3d576aa (0x20ab bytes) 2024-09-18T01:04:08.454 INFO:teuthology.orchestra.run.smithi164.stdout:5738 trunc from 0xb613400 to 0x8205c00 2024-09-18T01:04:08.468 INFO:teuthology.orchestra.run.smithi164.stdout:5739 write 0x22a3c00 thru 0x22a99a1 (0x5da2 bytes) 2024-09-18T01:04:08.484 INFO:teuthology.orchestra.run.smithi164.stdout:5740 write 0xb606a00 thru 0xb60940c (0x2a0d bytes) 2024-09-18T01:04:08.485 INFO:teuthology.orchestra.run.smithi164.stdout:5741 write 0x2711e00 thru 0x2719f40 (0x8141 bytes) 2024-09-18T01:04:08.489 INFO:teuthology.orchestra.run.smithi164.stdout:5742 punch from 0x821c00 to 0x82f000, (0xd400 bytes) 2024-09-18T01:04:08.489 INFO:teuthology.orchestra.run.smithi164.stdout:5743 punch from 0xe7600 to 0xf3400, (0xbe00 bytes) 2024-09-18T01:04:08.489 INFO:teuthology.orchestra.run.smithi164.stdout:5744 write 0x4a83000 thru 0x4a8afe7 (0x7fe8 bytes) 2024-09-18T01:04:08.490 INFO:teuthology.orchestra.run.smithi164.stdout:5746 read 0x92ec600 thru 0x92fc5bb (0xffbc bytes) 2024-09-18T01:04:08.490 INFO:teuthology.orchestra.run.smithi164.stdout:5749 read 0x8a46600 thru 0x8a4e17a (0x7b7b bytes) 2024-09-18T01:04:08.491 INFO:teuthology.orchestra.run.smithi164.stdout:5750 read 0x9dd7400 thru 0x9de566c (0xe26d bytes) 2024-09-18T01:04:08.492 INFO:teuthology.orchestra.run.smithi164.stdout:5751 read 0x3cf5a00 thru 0x3cf96e4 (0x3ce5 bytes) 2024-09-18T01:04:08.497 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:08.497 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:08.498 INFO:teuthology.orchestra.run.smithi164.stdout:5752 trunc from 0xb609600 to 0x7f15800 2024-09-18T01:04:08.506 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:08.514 INFO:teuthology.orchestra.run.smithi164.stdout:5755 punch from 0x5554000 to 0x5555e00, (0x1e00 bytes) 2024-09-18T01:04:08.516 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:08.515+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:08.517 INFO:teuthology.orchestra.run.smithi164.stdout:5756 read 0x27d600 thru 0x28adf5 (0xd7f6 bytes) 2024-09-18T01:04:08.533 INFO:teuthology.orchestra.run.smithi164.stdout:5762 write 0xdf82000 thru 0xdf87980 (0x5981 bytes) 2024-09-18T01:04:08.534 INFO:teuthology.orchestra.run.smithi164.stdout:5763 write 0xd405a00 thru 0xd40c1e2 (0x67e3 bytes) 2024-09-18T01:04:08.534 INFO:teuthology.orchestra.run.smithi164.stdout:5764 punch from 0xb103000 to 0xb104e00, (0x1e00 bytes) 2024-09-18T01:04:08.534 INFO:teuthology.orchestra.run.smithi164.stdout:5765 read 0x5430200 thru 0x54364ff (0x6300 bytes) 2024-09-18T01:04:08.535 INFO:teuthology.orchestra.run.smithi164.stdout:5768 read 0xdc68000 thru 0xdc6bbbd (0x3bbe bytes) 2024-09-18T01:04:08.536 INFO:teuthology.orchestra.run.smithi164.stdout:5774 write 0xd5dea00 thru 0xd5e9d06 (0xb307 bytes) 2024-09-18T01:04:08.536 INFO:teuthology.orchestra.run.smithi164.stdout:5776 read 0x89da000 thru 0x89e7e20 (0xde21 bytes) 2024-09-18T01:04:08.536 INFO:teuthology.orchestra.run.smithi164.stdout:5777 write 0xbf71e00 thru 0xbf74f0f (0x3110 bytes) 2024-09-18T01:04:08.536 INFO:teuthology.orchestra.run.smithi164.stdout:5781 write 0x80de000 thru 0x80e5c2d (0x7c2e bytes) 2024-09-18T01:04:08.536 INFO:teuthology.orchestra.run.smithi164.stdout:5782 read 0xab30000 thru 0xab3b059 (0xb05a bytes) 2024-09-18T01:04:08.536 INFO:teuthology.orchestra.run.smithi164.stdout:5783 write 0x9dea600 thru 0x9df21f1 (0x7bf2 bytes) 2024-09-18T01:04:08.536 INFO:teuthology.orchestra.run.smithi164.stdout:5784 read 0x1284c00 thru 0x1287349 (0x274a bytes) 2024-09-18T01:04:08.541 INFO:teuthology.orchestra.run.smithi164.stdout:5785 trunc from 0xdf87a00 to 0x9140e00 2024-09-18T01:04:08.567 INFO:teuthology.orchestra.run.smithi164.stdout:5789 read 0x746be00 thru 0x7476d07 (0xaf08 bytes) 2024-09-18T01:04:08.578 INFO:teuthology.orchestra.run.smithi164.stdout:5790 write 0x956f600 thru 0x957e78a (0xf18b bytes) 2024-09-18T01:04:08.579 INFO:teuthology.orchestra.run.smithi164.stdout:5791 punch from 0x7a1ae00 to 0x7a1ba00, (0xc00 bytes) 2024-09-18T01:04:08.580 INFO:teuthology.orchestra.run.smithi164.stdout:5792 punch from 0x8fe6e00 to 0x8fe9c00, (0x2e00 bytes) 2024-09-18T01:04:08.592 INFO:teuthology.orchestra.run.smithi164.stdout:5794 write 0xb230400 thru 0xb231727 (0x1328 bytes) 2024-09-18T01:04:08.592 INFO:teuthology.orchestra.run.smithi164.stdout:5796 read 0xad09000 thru 0xad1734f (0xe350 bytes) 2024-09-18T01:04:08.602 INFO:teuthology.orchestra.run.smithi164.stdout:5798 trunc from 0xb231800 to 0x3118200 2024-09-18T01:04:08.617 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:08.616+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:08.625 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:08.625 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:08.634 INFO:teuthology.orchestra.run.smithi164.stdout:5799 punch from 0x2b28000 to 0x2b36400, (0xe400 bytes) 2024-09-18T01:04:08.635 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:08.647 INFO:teuthology.orchestra.run.smithi164.stdout:5801 write 0x5043400 thru 0x50514ee (0xe0ef bytes) 2024-09-18T01:04:08.648 INFO:teuthology.orchestra.run.smithi164.stdout:5802 punch from 0x64a00 to 0x72800, (0xde00 bytes) 2024-09-18T01:04:08.649 INFO:teuthology.orchestra.run.smithi164.stdout:5804 write 0x1aa1600 thru 0x1aa2a5c (0x145d bytes) 2024-09-18T01:04:08.665 INFO:teuthology.orchestra.run.smithi164.stdout:5805 write 0x5170800 thru 0x5174770 (0x3f71 bytes) 2024-09-18T01:04:08.666 INFO:teuthology.orchestra.run.smithi164.stdout:5807 punch from 0x4122200 to 0x4127600, (0x5400 bytes) 2024-09-18T01:04:08.666 INFO:teuthology.orchestra.run.smithi164.stdout:5809 punch from 0xc9a00 to 0xcdc00, (0x4200 bytes) 2024-09-18T01:04:08.671 INFO:teuthology.orchestra.run.smithi164.stdout:5810 trunc from 0x5174800 to 0x1550600 2024-09-18T01:04:08.689 INFO:teuthology.orchestra.run.smithi164.stdout:5811 punch from 0xda2e00 to 0xdaa600, (0x7800 bytes) 2024-09-18T01:04:08.704 INFO:teuthology.orchestra.run.smithi164.stdout:5812 write 0x52a7400 thru 0x52ac0b8 (0x4cb9 bytes) 2024-09-18T01:04:08.717 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:08.716+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:08.721 INFO:teuthology.orchestra.run.smithi164.stdout:5816 write 0x9c07200 thru 0x9c0dca0 (0x6aa1 bytes) 2024-09-18T01:04:08.733 INFO:teuthology.orchestra.run.smithi164.stdout:5817 trunc from 0x9c0de00 to 0x69ce00 2024-09-18T01:04:08.754 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:08.755 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:08.762 INFO:teuthology.orchestra.run.smithi164.stdout:5819 read 0x5c2e00 thru 0x5cc1e5 (0x93e6 bytes) 2024-09-18T01:04:08.764 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:08.779 INFO:teuthology.orchestra.run.smithi164.stdout:5820 trunc from 0x69ce00 to 0xe573c00 2024-09-18T01:04:08.789 INFO:teuthology.orchestra.run.smithi164.stdout:5822 punch from 0x55d5e00 to 0x55e3c00, (0xde00 bytes) 2024-09-18T01:04:08.790 INFO:teuthology.orchestra.run.smithi164.stdout:5823 write 0x1935e00 thru 0x193f806 (0x9a07 bytes) 2024-09-18T01:04:08.791 INFO:teuthology.orchestra.run.smithi164.stdout:5824 read 0xc692e00 thru 0xc6a1638 (0xe839 bytes) 2024-09-18T01:04:08.791 INFO:teuthology.orchestra.run.smithi164.stdout:5825 write 0x6ff4600 thru 0x6ff50e0 (0xae1 bytes) 2024-09-18T01:04:08.791 INFO:teuthology.orchestra.run.smithi164.stdout:5826 punch from 0x8c9e400 to 0x8cad000, (0xec00 bytes) 2024-09-18T01:04:08.792 INFO:teuthology.orchestra.run.smithi164.stdout:5827 read 0x80e0400 thru 0x80ec5b1 (0xc1b2 bytes) 2024-09-18T01:04:08.792 INFO:teuthology.orchestra.run.smithi164.stdout:5829 write 0xe067200 thru 0xe07020a (0x900b bytes) 2024-09-18T01:04:08.805 INFO:teuthology.orchestra.run.smithi164.stdout:5830 trunc from 0xe573c00 to 0x2d08e00 2024-09-18T01:04:08.839 INFO:teuthology.orchestra.run.smithi164.stdout:5832 trunc from 0x2d08e00 to 0x2df5800 2024-09-18T01:04:08.848 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:08.847+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:08.850 INFO:teuthology.orchestra.run.smithi164.stdout:5833 read 0x2dbf200 thru 0x2dcaa51 (0xb852 bytes) 2024-09-18T01:04:08.859 INFO:teuthology.orchestra.run.smithi164.stdout:5834 trunc from 0x2df5800 to 0x98f8800 2024-09-18T01:04:08.868 INFO:teuthology.orchestra.run.smithi164.stdout:5840 read 0x18c6c00 thru 0x18c78c1 (0xcc2 bytes) 2024-09-18T01:04:08.875 INFO:teuthology.orchestra.run.smithi164.stdout:5844 trunc from 0x98f8800 to 0x4621200 2024-09-18T01:04:08.907 INFO:teuthology.orchestra.run.smithi164.stdout:5845 trunc from 0x4621200 to 0xe5aea00 2024-09-18T01:04:08.917 INFO:teuthology.orchestra.run.smithi164.stdout:5846 write 0xe475c00 thru 0xe47b88f (0x5c90 bytes) 2024-09-18T01:04:08.919 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:08.919 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:08.919 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:08.918+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:08.923 INFO:teuthology.orchestra.run.smithi164.stdout:5847 read 0xdb99c00 thru 0xdb9f635 (0x5a36 bytes) 2024-09-18T01:04:08.928 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:08.936 INFO:teuthology.orchestra.run.smithi164.stdout:5848 write 0xe974200 thru 0xe97adea (0x6beb bytes) 2024-09-18T01:04:08.943 INFO:teuthology.orchestra.run.smithi164.stdout:5850 read 0x510fc00 thru 0x511ee5f (0xf260 bytes) 2024-09-18T01:04:08.943 INFO:teuthology.orchestra.run.smithi164.stdout:5851 punch from 0xda3cc00 to 0xda41a00, (0x4e00 bytes) 2024-09-18T01:04:08.944 INFO:teuthology.orchestra.run.smithi164.stdout:5852 read 0x2ec3c00 thru 0x2ed291b (0xed1c bytes) 2024-09-18T01:04:08.944 INFO:teuthology.orchestra.run.smithi164.stdout:5853 read 0xd8d0000 thru 0xd8de263 (0xe264 bytes) 2024-09-18T01:04:08.944 INFO:teuthology.orchestra.run.smithi164.stdout:5854 punch from 0x7944200 to 0x794fa00, (0xb800 bytes) 2024-09-18T01:04:08.945 INFO:teuthology.orchestra.run.smithi164.stdout:5856 write 0xd88b800 thru 0xd89682e (0xb02f bytes) 2024-09-18T01:04:08.945 INFO:teuthology.orchestra.run.smithi164.stdout:5857 read 0x24e6e00 thru 0x24e9fff (0x3200 bytes) 2024-09-18T01:04:08.945 INFO:teuthology.orchestra.run.smithi164.stdout:5859 read 0x473fc00 thru 0x474e308 (0xe709 bytes) 2024-09-18T01:04:08.945 INFO:teuthology.orchestra.run.smithi164.stdout:5860 write 0x8fcae00 thru 0x8fd52ed (0xa4ee bytes) 2024-09-18T01:04:08.946 INFO:teuthology.orchestra.run.smithi164.stdout:5861 read 0xcfbe600 thru 0xcfc7330 (0x8d31 bytes) 2024-09-18T01:04:08.946 INFO:teuthology.orchestra.run.smithi164.stdout:5870 read 0x8e3c600 thru 0x8e45681 (0x9082 bytes) 2024-09-18T01:04:08.958 INFO:teuthology.orchestra.run.smithi164.stdout:5871 trunc from 0xe97ae00 to 0x3da9200 2024-09-18T01:04:09.001 INFO:teuthology.orchestra.run.smithi164.stdout:5872 write 0x990f800 thru 0x9911fc3 (0x27c4 bytes) 2024-09-18T01:04:09.001 INFO:teuthology.orchestra.run.smithi164.stdout:5876 read 0x34c600 thru 0x352a89 (0x648a bytes) 2024-09-18T01:04:09.002 INFO:teuthology.orchestra.run.smithi164.stdout:5877 punch from 0x4471e00 to 0x4481400, (0xf600 bytes) 2024-09-18T01:04:09.012 INFO:teuthology.orchestra.run.smithi164.stdout:5878 trunc from 0x9912000 to 0x14dd400 2024-09-18T01:04:09.020 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:09.019+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:09.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:09.056 INFO:teuthology.orchestra.run.smithi164.stdout:5879 trunc from 0x14dd400 to 0x9cee000 2024-09-18T01:04:09.065 INFO:teuthology.orchestra.run.smithi164.stdout:5881 read 0x2ba1600 thru 0x2bafa32 (0xe433 bytes) 2024-09-18T01:04:09.070 INFO:teuthology.orchestra.run.smithi164.stdout:5885 trunc from 0x9cee000 to 0xc9fbe00 2024-09-18T01:04:09.081 INFO:teuthology.orchestra.run.smithi164.stdout:5886 write 0x6804200 thru 0x680fada (0xb8db bytes) 2024-09-18T01:04:09.082 INFO:teuthology.orchestra.run.smithi164.stdout:5887 read 0x5613400 thru 0x5619f00 (0x6b01 bytes) 2024-09-18T01:04:09.082 INFO:teuthology.orchestra.run.smithi164.stdout:5891 read 0x86d8400 thru 0x86df455 (0x7056 bytes) 2024-09-18T01:04:09.082 INFO:teuthology.orchestra.run.smithi164.stdout:5893 read 0xc6e6800 thru 0xc6f5f4f (0xf750 bytes) 2024-09-18T01:04:09.082 INFO:teuthology.orchestra.run.smithi164.stdout:5894 write 0x1c21000 thru 0x1c299bd (0x89be bytes) 2024-09-18T01:04:09.083 INFO:teuthology.orchestra.run.smithi164.stdout:5897 write 0x22e0600 thru 0x22ea846 (0xa247 bytes) 2024-09-18T01:04:09.083 INFO:teuthology.orchestra.run.smithi164.stdout:5902 write 0x4691400 thru 0x469789c (0x649d bytes) 2024-09-18T01:04:09.084 INFO:teuthology.orchestra.run.smithi164.stdout:5904 read 0x2fe6e00 thru 0x2ff00ef (0x92f0 bytes) 2024-09-18T01:04:09.086 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:09.086 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:09.090 INFO:teuthology.orchestra.run.smithi164.stdout:5905 trunc from 0xc9fbe00 to 0x750d000 2024-09-18T01:04:09.096 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:09.102 INFO:teuthology.orchestra.run.smithi164.stdout:5906 read 0x265a200 thru 0x26626d4 (0x84d5 bytes) 2024-09-18T01:04:09.104 INFO:teuthology.orchestra.run.smithi164.stdout:5911 read 0x6625200 thru 0x66311af (0xbfb0 bytes) 2024-09-18T01:04:09.104 INFO:teuthology.orchestra.run.smithi164.stdout:5912 punch from 0x5b12000 to 0x5b16e00, (0x4e00 bytes) 2024-09-18T01:04:09.107 INFO:teuthology.orchestra.run.smithi164.stdout:5915 trunc from 0x750d000 to 0x55a0400 2024-09-18T01:04:09.121 INFO:teuthology.orchestra.run.smithi164.stdout:5916 write 0x27d3000 thru 0x27ddd07 (0xad08 bytes) 2024-09-18T01:04:09.122 INFO:teuthology.orchestra.run.smithi164.stdout:5919 punch from 0x4e18c00 to 0x4e20800, (0x7c00 bytes) 2024-09-18T01:04:09.137 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:09.136+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:09.137 INFO:teuthology.orchestra.run.smithi164.stdout:5922 write 0x89b3000 thru 0x89b39d4 (0x9d5 bytes) 2024-09-18T01:04:09.139 INFO:teuthology.orchestra.run.smithi164.stdout:5923 trunc from 0x89b3a00 to 0x6eb5400 2024-09-18T01:04:09.165 INFO:teuthology.orchestra.run.smithi164.stdout:5924 punch from 0x1a41400 to 0x1a4c400, (0xb000 bytes) 2024-09-18T01:04:09.166 INFO:teuthology.orchestra.run.smithi164.stdout:5926 read 0x12aec00 thru 0x12b0897 (0x1c98 bytes) 2024-09-18T01:04:09.167 INFO:teuthology.orchestra.run.smithi164.stdout:5928 read 0x47ee400 thru 0x47f916e (0xad6f bytes) 2024-09-18T01:04:09.167 INFO:teuthology.orchestra.run.smithi164.stdout:5929 read 0x5bc7000 thru 0x5bcff17 (0x8f18 bytes) 2024-09-18T01:04:09.170 INFO:teuthology.orchestra.run.smithi164.stdout:5933 punch from 0x6f5400 to 0x6fde00, (0x8a00 bytes) 2024-09-18T01:04:09.170 INFO:teuthology.orchestra.run.smithi164.stdout:5934 write 0x3a50200 thru 0x3a5e97d (0xe77e bytes) 2024-09-18T01:04:09.189 INFO:teuthology.orchestra.run.smithi164.stdout:5937 write 0xc746e00 thru 0xc74cd45 (0x5f46 bytes) 2024-09-18T01:04:09.195 INFO:teuthology.orchestra.run.smithi164.stdout:5940 punch from 0x33da600 to 0x33e2c00, (0x8600 bytes) 2024-09-18T01:04:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:5941 write 0xc649400 thru 0xc64d1f2 (0x3df3 bytes) 2024-09-18T01:04:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:5942 write 0x8304800 thru 0x83114bc (0xccbd bytes) 2024-09-18T01:04:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:5948 read 0xb650e00 thru 0xb65f319 (0xe51a bytes) 2024-09-18T01:04:09.196 INFO:teuthology.orchestra.run.smithi164.stdout:5949 read 0xb140a00 thru 0xb14a733 (0x9d34 bytes) 2024-09-18T01:04:09.205 INFO:teuthology.orchestra.run.smithi164.stdout:5950 trunc from 0xc74ce00 to 0x51ec600 2024-09-18T01:04:09.222 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:09.221+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:09.237 INFO:teuthology.orchestra.run.smithi164.stdout:5951 punch from 0x1945c00 to 0x194e600, (0x8a00 bytes) 2024-09-18T01:04:09.244 INFO:teuthology.orchestra.run.smithi164.stdout:5954 read 0xc08000 thru 0xc13a70 (0xba71 bytes) 2024-09-18T01:04:09.245 INFO:teuthology.orchestra.run.smithi164.stdout:5956 write 0x3927e00 thru 0x39341f0 (0xc3f1 bytes) 2024-09-18T01:04:09.246 INFO:teuthology.orchestra.run.smithi164.stdout:5958 read 0x25f8000 thru 0x25fbc03 (0x3c04 bytes) 2024-09-18T01:04:09.246 INFO:teuthology.orchestra.run.smithi164.stdout:5959 read 0x27c5e00 thru 0x27c60ab (0x2ac bytes) 2024-09-18T01:04:09.249 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:09.249 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:09.258 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:09.264 INFO:teuthology.orchestra.run.smithi164.stdout:5962 write 0x92dce00 thru 0x92ec4ae (0xf6af bytes) 2024-09-18T01:04:09.281 INFO:teuthology.orchestra.run.smithi164.stdout:5964 trunc from 0x92ec600 to 0x84b200 2024-09-18T01:04:09.314 INFO:teuthology.orchestra.run.smithi164.stdout:5965 read 0x517e00 thru 0x521b5f (0x9d60 bytes) 2024-09-18T01:04:09.323 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:09.322+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:09.327 INFO:teuthology.orchestra.run.smithi164.stdout:5966 trunc from 0x84b200 to 0x5d2f200 2024-09-18T01:04:09.338 INFO:teuthology.orchestra.run.smithi164.stdout:5967 write 0x53d4200 thru 0x53dc036 (0x7e37 bytes) 2024-09-18T01:04:09.351 INFO:teuthology.orchestra.run.smithi164.stdout:5968 write 0x6920800 thru 0x692aeaf (0xa6b0 bytes) 2024-09-18T01:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout:5970 read 0x65f1c00 thru 0x65f9567 (0x7968 bytes) 2024-09-18T01:04:09.352 INFO:teuthology.orchestra.run.smithi164.stdout:5971 write 0x4fde00 thru 0x502b7c (0x4d7d bytes) 2024-09-18T01:04:09.353 INFO:teuthology.orchestra.run.smithi164.stdout:5976 write 0x4981600 thru 0x498f829 (0xe22a bytes) 2024-09-18T01:04:09.354 INFO:teuthology.orchestra.run.smithi164.stdout:5977 punch from 0x397b200 to 0x397d200, (0x2000 bytes) 2024-09-18T01:04:09.362 INFO:teuthology.orchestra.run.smithi164.stdout:5978 trunc from 0x692b000 to 0x295600 2024-09-18T01:04:09.390 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:09.390 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:09.408 INFO:teuthology.orchestra.run.smithi164.stdout:5982 write 0x555fe00 thru 0x5562b52 (0x2d53 bytes) 2024-09-18T01:04:09.409 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:09.409 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:09.415 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:09.416 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:09.424 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:09.423+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:09.424 INFO:teuthology.orchestra.run.smithi164.stdout:5983 write 0x9c19e00 thru 0x9c24dee (0xafef bytes) 2024-09-18T01:04:09.426 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:09.426 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:09.427 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:09.435 INFO:teuthology.orchestra.run.smithi164.stdout:5988 trunc from 0x9c24e00 to 0x8f4c00 2024-09-18T01:04:09.447 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:09.447 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:09.467 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:09.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:09.484 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:09.488 INFO:teuthology.orchestra.run.smithi164.stdout:5989 trunc from 0x8f4c00 to 0xd344800 2024-09-18T01:04:09.501 INFO:teuthology.orchestra.run.smithi164.stdout:5990 read 0x28e3400 thru 0x28f3131 (0xfd32 bytes) 2024-09-18T01:04:09.517 INFO:teuthology.orchestra.run.smithi164.stdout:5994 write 0xdbea800 thru 0xdbf135e (0x6b5f bytes) 2024-09-18T01:04:09.518 INFO:teuthology.orchestra.run.smithi164.stdout:5995 punch from 0x4031a00 to 0x4034e00, (0x3400 bytes) 2024-09-18T01:04:09.518 INFO:teuthology.orchestra.run.smithi164.stdout:5999 punch from 0x97a0a00 to 0x97a7400, (0x6a00 bytes) 2024-09-18T01:04:09.536 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:09.535+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:09.539 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:09.539 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:09.539 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:09.539 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:09.548 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:09.626 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:09.625+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:09.653 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:09.653 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:09.653 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:09.653 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:09.653 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:09.662 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:09.695 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-18T01:04:09.727 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:09.726+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:09.772 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 97.396068094, 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.398056788999995, 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 95.390379836999998, 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:09.788 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:09.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:09.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:09.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:09.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.795 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.292521887000007, 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.796 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:09.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.801 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.802 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.436440875000002, 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.803 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:09.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.805 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:09.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:09.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.808 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.436071579, 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.809 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:09.810 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.811 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:09.812 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.813 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.814 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.434332733999995, 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:09.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.816 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.817 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.818 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.434260843999994, 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.819 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.820 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.821 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.822 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.822 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:09.824 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.346394833999994, 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.827 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:09.828 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.829 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:09.830 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.831 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:09.832 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:09.833 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.340203575000004, 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.835 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:09.836 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.837 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.338673889999995, 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.839 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:09.840 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.841 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:09.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.843 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.844 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.338294200999997, 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.845 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:09.846 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.847 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.848 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.850 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.322795798000001, 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.851 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.853 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:09.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.855 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:09.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.857 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.322589038999993, 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.858 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.859 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:09.860 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.862 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:09.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.322171281999999, 2024-09-18T01:04:09.864 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:09.865 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.866 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:09.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:09.868 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.321962044000003, 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:09.871 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.872 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:09.873 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:09.874 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.321898110000006, 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.876 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.877 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:09.878 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.321149904999999, 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.879 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.881 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.882 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:09.883 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.062211099999999, 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.884 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.885 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:09.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:09.887 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:09.889 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.061347482000002, 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:09.890 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:09.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.892 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:09.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:09.894 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:09.895 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:09.897 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:09.895+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:09.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:09.998 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:09.997+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:10.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:10.005 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:10.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:10.098 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:10.098+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:10.160 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:10.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:10.188 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:04:10.188 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:04:10.188 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-18T01:04:10.188 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:04:10.188 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:04:10.188 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-18T01:04:10.209 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:10.208+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:10.292 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:10.292 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T01:04:10.292 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:10.301 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:10.299+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:10.316 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.996333665999998, 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.996313157000003, 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.996293351999995, 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.996258711999999, 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.996237210999993, 2024-09-18T01:04:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.996214034000005, 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992225202, 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992205648999999, 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992185144999993, 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992164281000001, 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992144381000003, 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992120292999999, 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992088972999994, 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992044367000005, 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.992013576000005, 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.991990091000005, 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.991961036000006, 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.991940276999998, 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.733999529999998, 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.733915821, 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.413 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:10.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:10.415 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:10.413+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:10.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:04:10.515 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:10.513+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:10.577 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:10.578 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:10.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:10.616 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:10.614+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:10.697 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:10.698 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:10.698 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:10.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:10.717 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:10.716+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:10.815 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.815 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.536041495999996, 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:10.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:10.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.531415593000006, 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:10.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:10.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:10.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:10.830 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.531337543999996, 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:10.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:10.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:10.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:10.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.474541501000004, 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:10.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:10.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.474515636999996, 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:10.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.474493156999998, 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:10.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.474472121000005, 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.474444727000005, 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.474420421999994, 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.471020938999999, 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:04:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470996378999999, 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470954861999999, 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470934360000001, 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470913894000006, 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470872553999996, 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470852515999994, 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470810185000005, 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470789193000002, 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.470738222999998, 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.238597835, 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:10.918 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:10.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:10.921 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:10.918+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:11.020 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:11.019+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:11.027 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:11.027 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:11.027 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:11.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:11.121 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:11.119+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:11.183 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:11.183 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:11.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:11.235 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:11.234+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:11.300 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.563817612999998, 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:11.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.563797655000002, 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:11.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.313 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.563778713999994, 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:11.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:11.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.563757205000002, 2024-09-18T01:04:11.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:11.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:11.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.563731477000005, 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:11.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:11.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554999617999997, 2024-09-18T01:04:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:11.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:04:11.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554976904, 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554957774000002, 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554938927999999, 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554920050000007, 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554898184999999, 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554874549999994, 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554840900000002, 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554813045000003, 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554793222000001, 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554772787999994, 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554750708, 2024-09-18T01:04:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554728385999994, 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554705990000002, 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.554685570999993, 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:11.430 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:11.432 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:11.432 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:11.430+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:04:11.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:11.532 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:11.530+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:11.546 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:11.546 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:11.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4355.0:22000 3.4 e90/75 3:21e557b5:::rbd_object_map.1103735b3cf3:head v 90'439, mlcod=90'437)", 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:04:11.544468+0000", 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 0.0018322779999999999, 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00184856, 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544468+0000", 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:11.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544468+0000", 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544468+0000", 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3200000000000005e-07 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:11.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544472+0000", 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7390000000000001e-06 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544482+0000", 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5599999999999999e-06 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544486+0000", 2024-09-18T01:04:11.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5549999999999996e-06 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544532+0000", 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5862999999999998e-05 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:11.544546+0000", 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4276e-05 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:11.550 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:11.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:11.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:11.551 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-18T01:04:11.551 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 1 2024-09-18T01:04:11.551 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:11.556 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:11.629 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:11.629 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:11.633 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:11.633+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:11.646 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:11.646 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:11.662 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:11.662 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:11.679 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:11.680 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:11.696 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:11.696 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:11.700 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:11.700 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:11.709 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:11.712 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:11.753 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:11.751+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:11.812 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:11.812 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:11.821 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:11.862 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:11.861+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:11.936 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:11.933+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:11.965 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:11.965 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:11.974 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:12.037 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:12.036+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:12.126 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:12.169 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:12.167+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:12.228 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:12.228 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:12.236 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:12.277 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:12.275+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:12.340 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:12.337+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:12.381 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:12.381 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:12.389 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:12.474 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:12.472+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:12.542 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:12.540+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:12.544 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:12.544 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:12.557 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:12.643 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:12.641+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:12.700 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:12.701 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:12.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:12.753 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:12.752+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:12.817 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:12.817 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:12.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:12.867 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:12.866+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:12.946 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:12.945+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:12.973 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:12.973 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:12.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:13.048 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:13.046+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:13.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:13.149 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:13.147+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:13.204 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:13.205 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:13.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:13.258 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:13.257+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:13.320 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:13.320 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:13.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:13.370 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:13.369+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:13.451 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:13.449+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:13.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:04:13.485 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:13.485 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:13.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:13.581 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:13.580+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:13.650 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:13.651 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:13.653 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:13.652+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:13.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:13.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:13.787 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:13.786+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:13.810 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:13.810 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:13.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:13.860 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:13.859+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:13.869 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:13.869 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:13.887 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:13.887 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:13.903 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:13.903 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:13.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:13.938 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:13.938 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:13.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:13.955 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:13.956 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:13.953+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:13.969 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.232869245000003, 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.232849286999993, 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.232830346, 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.232808836999993, 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.232783108999996, 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.224051250000002, 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:04:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:04:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.224028536000006, 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.224009405999993, 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223990560000004, 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223971681999998, 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223949817000005, 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223926182, 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223892531999994, 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223864676999995, 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223844854000006, 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.22382442, 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223802340000006, 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223780017999999, 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223757621999994, 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.223737202999999, 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:14.092 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:14.093 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:14.094 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:14.092+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:14.194 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:14.193+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:14.208 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:14.208 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:14.208 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:14.217 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:14.295 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:14.293+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:14.319 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:14.320 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:14.320 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:14.320 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:14.320 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:14.328 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:14.396 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:14.395+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:14.465 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:04:14.480 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:14.480 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:14.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:14.481 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:14.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.481 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.481 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:14.481 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.433747561999994, 2024-09-18T01:04:14.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:04:14.482 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.482 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.482 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.482 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.482 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.483 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.483 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:04:14.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.483 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:14.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:14.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:14.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:04:14.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:04:14.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:04:14.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:04:14.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:04:14.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:04:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:04:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.433736410999998, 2024-09-18T01:04:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:04:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.493 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:04:14.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:14.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:04:14.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:04:14.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.499 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.433724581000007, 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.500 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:04:14.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.433707897999994, 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.505 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:04:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:04:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:04:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:04:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:04:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:04:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:04:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:04:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:04:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:04:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:04:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:04:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:04:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.514 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:14.515 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.433696660999999, 2024-09-18T01:04:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:04:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:04:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:04:14.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:04:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:04:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:04:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:04:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:04:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:04:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:04:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:04:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:04:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:04:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:04:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:04:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:04:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.432374555999999, 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:04:14.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:04:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:14.533 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.432361841000002, 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:04:14.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.432316857000004, 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.538 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:04:14.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.432289233000006, 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:04:14.543 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:04:14.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.547 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.428741853999995, 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:04:14.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.428381747000003, 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.553 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:04:14.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.428355633999999, 2024-09-18T01:04:14.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:14.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:04:14.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.426176245999997, 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.566 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.426166451, 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:14.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:04:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:04:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:04:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:04:14.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.574 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.426155671999993, 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:14.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:04:14.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.581 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.426135876000004, 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:04:14.582 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:14.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:04:14.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:04:14.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.424352356, 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.589 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:04:14.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.593 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.424325518000003, 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.594 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:04:14.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.598 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 72.424314969999998, 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:04:14.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:04:14.603 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 36.952388094, 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:04:14.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:04:14.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:14.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:04:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:04:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:14.609 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:14.609 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:14.612 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:14.613 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:14.608+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:14.712 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:14.711+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:14.754 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:14.754 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:14.755 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:14.755 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:14.764 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:14.813 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:14.811+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:14.867 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:14.867 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:14.867 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:14.868 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:14.868 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:14.875 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:14.917 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:14.916+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:15.015 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:15.013+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:15.020 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:15.020 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:15.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:15.020 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:15.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.020 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.722824063999994, 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.021 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:15.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:15.024 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.722799946999999, 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.025 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:15.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:15.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.029 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.722720706999993, 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.030 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.034 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.722684487999999, 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:15.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.038 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.722655875000001, 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.039 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.043 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.722635010000005, 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.044 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:15.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.722017292999993, 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.049 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.053 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721988026000005, 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.057 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721921734999995, 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.058 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:15.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721800967999997, 2024-09-18T01:04:15.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:15.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721582522000006, 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:15.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721561160999997, 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.071 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:15.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:15.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.076 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721407221999996, 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:15.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:15.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721387504000006, 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.083 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:15.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.088 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721367193999995, 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:15.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.092 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721347308000006, 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.093 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:15.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721327349000006, 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.097 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:15.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.101 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721307350999993, 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:15.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721287684000004, 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:15.106 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:15.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:15.110 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 68.721267781999998, 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:15.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:15.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.115 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.116 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.116 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:15.117 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:15.118 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:15.116+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:15.218 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:15.216+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:15.260 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:15.260 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:15.260 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:15.260 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:15.269 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:15.301 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:04:15.302 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:04:15.302 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:04:15.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:04:15.349 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:15.348+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:15.411 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:15.411 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:15.411 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:15.412 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:15.412 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:15.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:15.420 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:15.418+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:15.519 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:15.519+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:15.533 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:15.533 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 103.15704432299999, 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.534 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:15.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:15.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:15.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:15.540 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.159033018, 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:15.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:15.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.547 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.15135606600001, 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:15.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:15.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:15.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 92.053498116, 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:15.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:15.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:15.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.564 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.197417103999996, 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.565 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:15.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:15.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.197047807999994, 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.571 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:15.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:15.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.195308963000002, 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:15.577 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:15.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:15.581 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.195237073000001, 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:15.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:15.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.107371063000002, 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.586 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:15.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:15.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.101179803999997, 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.592 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:15.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:15.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.596 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.099650119000003, 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.597 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:15.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:15.602 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.099270430000004, 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.603 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:15.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:15.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.083772026999995, 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.609 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:15.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:15.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.083565268000001, 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.615 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:15.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:15.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:15.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.083147511000007, 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:15.621 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:15.625 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.082938272999996, 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:15.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:15.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.082874339, 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:15.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 88.082126134000006, 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:15.635 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:15.639 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 87.823187329000007, 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.640 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:15.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:15.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 87.822323710999996, 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:15.646 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:15.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:15.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:15.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:15.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:15.652 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:15.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 94, 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-18T01:02:13.493841+0000", 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-18T01:04:15.459995+0000", 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-18T01:02:24.290620+0000", 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-18T01:02:48.274416+0000", 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-18T01:04:15.655 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:04:15.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:24.388064+0000", 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:15.657 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:15.658 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:15.659 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:15.660 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:15.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.662 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.663 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:30.676199+0000", 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:15.664 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:15.665 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:15.666 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:15.667 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:15.668 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2699999809265137, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2699999809265137, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:40.390732+0000", 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "94", 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 33, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 94, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.7000000476837158, 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.7000000476837158, 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:04:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4563fb16-8745-4e75-b654-1e96403d91b9", 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.8100128173828125, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/1626166637", 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/1626166637", 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/1626166637", 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/1626166637", 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "634965cb-d444-4721-b34a-afbc49105cab", 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/2210970259", 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/2210970259", 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/2210970259", 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/2210970259", 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "83443eab-2fa4-4c7e-a459-edb1bb0ed8d0", 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.6881561279296875, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/617133675", 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/617133675", 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/617133675", 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/617133675", 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "5797982a-71c1-4d6a-9604-00fbc6bd16d9", 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6808", 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6809", 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6810", 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6811", 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6814", 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6815", 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6812", 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6813", 2024-09-18T01:04:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6809/151031464", 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6811/151031464", 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6815/151031464", 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6813/151031464", 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f79e3329-5481-4f33-8d28-6386af186492", 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 74, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6816", 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6817", 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6818", 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6819", 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6822", 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6823", 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6820", 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6821", 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6817/1499036965", 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6819/1499036965", 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6823/1499036965", 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6821/1499036965", 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "8c61d2ec-b111-4783-a016-336ed33a0694", 2024-09-18T01:04:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 20, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6800", 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6801", 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6802", 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6803", 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6806", 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6807", 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6804", 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6805", 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6801/2310498524", 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6803/2310498524", 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6807/2310498524", 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6805/2310498524", 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:04:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.877700+0000", 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.360930+0000", 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.403205+0000", 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.952123+0000", 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:04:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:22.099545+0000", 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.694132+0000", 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-18T01:04:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 28, 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 6 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 28, 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:15.720 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:15.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:15.722 INFO:tasks.thrashosds.thrasher:j is {'epoch': 94, 'fsid': '62b46b83-8ef2-466f-b28b-2d97f265422e', 'created': '2024-09-18T01:02:13.493841+0000', 'modified': '2024-09-18T01:04:15.459995+0000', 'last_up_change': '2024-09-18T01:02:24.290620+0000', 'last_in_change': '2024-09-18T01:02:48.274416+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:02:24.388064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:02:30.676199+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:02:40.390732+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4563fb16-8745-4e75-b654-1e96403d91b9', 'up': 1, 'in': 1, 'weight': 0.8100128173828125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 1626166637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 1626166637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 1626166637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 1626166637}]}, 'public_addr': '172.21.15.31:6817/1626166637', 'cluster_addr': '172.21.15.31:6819/1626166637', 'heartbeat_back_addr': '172.21.15.31:6823/1626166637', 'heartbeat_front_addr': '172.21.15.31:6821/1626166637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '634965cb-d444-4721-b34a-afbc49105cab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 2210970259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 2210970259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 2210970259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 2210970259}]}, 'public_addr': '172.21.15.31:6801/2210970259', 'cluster_addr': '172.21.15.31:6803/2210970259', 'heartbeat_back_addr': '172.21.15.31:6807/2210970259', 'heartbeat_front_addr': '172.21.15.31:6805/2210970259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'up': 1, 'in': 1, 'weight': 0.6881561279296875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 617133675}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 617133675}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 617133675}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 617133675}]}, 'public_addr': '172.21.15.31:6809/617133675', 'cluster_addr': '172.21.15.31:6811/617133675', 'heartbeat_back_addr': '172.21.15.31:6815/617133675', 'heartbeat_front_addr': '172.21.15.31:6813/617133675', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6808', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6809', 'nonce': 151031464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6810', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6811', 'nonce': 151031464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6814', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6815', 'nonce': 151031464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6812', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6813', 'nonce': 151031464}]}, 'public_addr': '172.21.15.98:6809/151031464', 'cluster_addr': '172.21.15.98:6811/151031464', 'heartbeat_back_addr': '172.21.15.98:6815/151031464', 'heartbeat_front_addr': '172.21.15.98:6813/151031464', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f79e3329-5481-4f33-8d28-6386af186492', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6816', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6817', 'nonce': 1499036965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6818', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6819', 'nonce': 1499036965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6822', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6823', 'nonce': 1499036965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6820', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6821', 'nonce': 1499036965}]}, 'public_addr': '172.21.15.98:6817/1499036965', 'cluster_addr': '172.21.15.98:6819/1499036965', 'heartbeat_back_addr': '172.21.15.98:6823/1499036965', 'heartbeat_front_addr': '172.21.15.98:6821/1499036965', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8c61d2ec-b111-4783-a016-336ed33a0694', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6800', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6801', 'nonce': 2310498524}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6802', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6803', 'nonce': 2310498524}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6806', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6807', 'nonce': 2310498524}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6804', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6805', 'nonce': 2310498524}]}, 'public_addr': '172.21.15.98:6801/2310498524', 'cluster_addr': '172.21.15.98:6803/2310498524', 'heartbeat_back_addr': '172.21.15.98:6807/2310498524', 'heartbeat_front_addr': '172.21.15.98:6805/2310498524', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.877700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.360930+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.403205+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.952123+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:22.099545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.694132+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 28, 'length': 6}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 28, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:04:15.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:04:15.723 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:15.653+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:15.757 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:15.756+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:15.834 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:15.835 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:15.835 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:15.835 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:15.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:15.859 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:15.857+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:15.953 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:15.954 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:15.959 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:15.958+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:15.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:15.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:16.019 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:04:16.032 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-18T01:04:15.461956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1027697210,"num_objects":1123,"num_object_clones":272,"num_object_copies":2246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1123,"num_whiteouts":400,"num_read":258251,"num_read_kb":1194196,"num_write":21721,"num_write_kb":288487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":29195562,"num_keys_recovered":73,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18699,"ondisk_log_size":18699,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":360180,"kb_used_data":44564,"kb_used_omap":7,"kb_used_meta":315448,"kb_avail":471499020,"statfs":{"total":483183820800,"available":482814996480,"internally_reserved":0,"allocated":45633536,"data_stored":313302656,"data_compressed":7760264,"data_compressed_allocated":24928256,"data_compressed_original":298327040,"omap_allocated":7973,"internal_metadata":323018971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-100233314,"num_objects":-202,"num_object_clones":-45,"num_object_copies":-404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-202,"num_whiteouts":-22,"num_read":2437,"num_read_kb":6042,"num_write":369,"num_write_kb":214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020416"},"pg_stats":[{"pgid":"3.1f","version":"92'630","reported_seq":10115,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.470854+0000","last_change":"2024-09-18T01:04:13.471062+0000","last_active":"2024-09-18T01:04:13.470854+0000","last_peered":"2024-09-18T01:04:13.470854+0000","last_clean":"2024-09-18T01:04:13.470854+0000","last_became_active":"2024-09-18T01:03:45.291862+0000","last_became_peered":"2024-09-18T01:03:45.291862+0000","last_unstale":"2024-09-18T01:04:13.470854+0000","last_undegraded":"2024-09-18T01:04:13.470854+0000","last_fullsized":"2024-09-18T01:04:13.470854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:18.871049+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.016419336999999999,"stat_sum":{"num_bytes":43588112,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":12632,"num_read_kb":58132,"num_write":531,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2632208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.1e","version":"92'1400","reported_seq":7351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.622828+0000","last_change":"2024-09-18T01:04:13.454563+0000","last_active":"2024-09-18T01:04:13.622828+0000","last_peered":"2024-09-18T01:04:13.622828+0000","last_clean":"2024-09-18T01:04:13.622828+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:04:13.622828+0000","last_undegraded":"2024-09-18T01:04:13.622828+0000","last_fullsized":"2024-09-18T01:04:13.622828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:50:12.934484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013017370000000001,"stat_sum":{"num_bytes":26395887,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":12024,"num_read_kb":29048,"num_write":1956,"num_write_kb":8237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.1d","version":"91'394","reported_seq":7067,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.461650+0000","last_change":"2024-09-18T01:04:13.462261+0000","last_active":"2024-09-18T01:04:13.461650+0000","last_peered":"2024-09-18T01:04:13.461650+0000","last_clean":"2024-09-18T01:04:13.461650+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:04:13.461650+0000","last_undegraded":"2024-09-18T01:04:13.461650+0000","last_fullsized":"2024-09-18T01:04:13.461650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:25.234338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003158906,"stat_sum":{"num_bytes":20717280,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":7352,"num_read_kb":28949,"num_write":392,"num_write_kb":8542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.1c","version":"92'768","reported_seq":12254,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.530988+0000","last_change":"2024-09-18T01:04:13.461584+0000","last_active":"2024-09-18T01:04:13.530988+0000","last_peered":"2024-09-18T01:04:13.530988+0000","last_clean":"2024-09-18T01:04:13.530988+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:04:13.530988+0000","last_undegraded":"2024-09-18T01:04:13.530988+0000","last_fullsized":"2024-09-18T01:04:13.530988+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:21:55.491294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024591320000000002,"stat_sum":{"num_bytes":40775310,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":15453,"num_read_kb":43200,"num_write":636,"num_write_kb":8638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.1b","version":"92'557","reported_seq":4158,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.461775+0000","last_change":"2024-09-18T01:04:13.462241+0000","last_active":"2024-09-18T01:04:13.461775+0000","last_peered":"2024-09-18T01:04:13.461775+0000","last_clean":"2024-09-18T01:04:13.461775+0000","last_became_active":"2024-09-18T01:03:44.290192+0000","last_became_peered":"2024-09-18T01:03:44.290192+0000","last_unstale":"2024-09-18T01:04:13.461775+0000","last_undegraded":"2024-09-18T01:04:13.461775+0000","last_fullsized":"2024-09-18T01:04:13.461775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:39:16.891732+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003183852,"stat_sum":{"num_bytes":18773636,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":4907,"num_read_kb":19402,"num_write":729,"num_write_kb":5919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":1672836,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.1a","version":"91'221","reported_seq":5596,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.599567+0000","last_change":"2024-09-18T01:04:13.460637+0000","last_active":"2024-09-18T01:04:13.599567+0000","last_peered":"2024-09-18T01:04:13.599567+0000","last_clean":"2024-09-18T01:04:13.599567+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:04:13.599567+0000","last_undegraded":"2024-09-18T01:04:13.599567+0000","last_fullsized":"2024-09-18T01:04:13.599567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:32.378995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015349350000000001,"stat_sum":{"num_bytes":13274352,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":5240,"num_read_kb":19224,"num_write":221,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.19","version":"91'572","reported_seq":8242,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.462489+0000","last_change":"2024-09-18T01:04:13.462733+0000","last_active":"2024-09-18T01:04:13.462489+0000","last_peered":"2024-09-18T01:04:13.462489+0000","last_clean":"2024-09-18T01:04:13.462489+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:04:13.462489+0000","last_undegraded":"2024-09-18T01:04:13.462489+0000","last_fullsized":"2024-09-18T01:04:13.462489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:50:16.996195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025966800000000001,"stat_sum":{"num_bytes":28103769,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":10561,"num_read_kb":39729,"num_write":549,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.18","version":"91'673","reported_seq":8238,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.453415+0000","last_change":"2024-09-18T01:04:13.453714+0000","last_active":"2024-09-18T01:04:13.453415+0000","last_peered":"2024-09-18T01:04:13.453415+0000","last_clean":"2024-09-18T01:04:13.453415+0000","last_became_active":"2024-09-18T01:03:45.291065+0000","last_became_peered":"2024-09-18T01:03:45.291065+0000","last_unstale":"2024-09-18T01:04:13.453415+0000","last_undegraded":"2024-09-18T01:04:13.453415+0000","last_fullsized":"2024-09-18T01:04:13.453415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:47.241353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074264299999999997,"stat_sum":{"num_bytes":63039029,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":9024,"num_read_kb":82819,"num_write":663,"num_write_kb":22267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":11170357,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.17","version":"92'846","reported_seq":7938,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.621895+0000","last_change":"2024-09-18T01:04:13.460265+0000","last_active":"2024-09-18T01:04:13.621895+0000","last_peered":"2024-09-18T01:04:13.621895+0000","last_clean":"2024-09-18T01:04:13.621895+0000","last_became_active":"2024-09-18T01:03:45.291814+0000","last_became_peered":"2024-09-18T01:03:45.291814+0000","last_unstale":"2024-09-18T01:04:13.621895+0000","last_undegraded":"2024-09-18T01:04:13.621895+0000","last_fullsized":"2024-09-18T01:04:13.621895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:17.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051090229999999999,"stat_sum":{"num_bytes":44564045,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":10491,"num_read_kb":32266,"num_write":1015,"num_write_kb":6699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1702477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.16","version":"92'329","reported_seq":3600,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.462427+0000","last_change":"2024-09-18T01:04:13.463050+0000","last_active":"2024-09-18T01:04:13.462427+0000","last_peered":"2024-09-18T01:04:13.462427+0000","last_clean":"2024-09-18T01:04:13.462427+0000","last_became_active":"2024-09-18T01:03:38.521858+0000","last_became_peered":"2024-09-18T01:03:38.521858+0000","last_unstale":"2024-09-18T01:04:13.462427+0000","last_undegraded":"2024-09-18T01:04:13.462427+0000","last_fullsized":"2024-09-18T01:04:13.462427+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:44.749916+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0087317990000000002,"stat_sum":{"num_bytes":19201198,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":3174,"num_read_kb":32404,"num_write":388,"num_write_kb":7748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.15","version":"91'519","reported_seq":9442,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.600152+0000","last_change":"2024-09-18T01:04:13.456038+0000","last_active":"2024-09-18T01:04:13.600152+0000","last_peered":"2024-09-18T01:04:13.600152+0000","last_clean":"2024-09-18T01:04:13.600152+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:04:13.600152+0000","last_undegraded":"2024-09-18T01:04:13.600152+0000","last_fullsized":"2024-09-18T01:04:13.600152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:05:07.149531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017401929999999999,"stat_sum":{"num_bytes":24108677,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":11284,"num_read_kb":41104,"num_write":409,"num_write_kb":4757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.14","version":"91'512","reported_seq":4944,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.461381+0000","last_change":"2024-09-18T01:04:13.462001+0000","last_active":"2024-09-18T01:04:13.461381+0000","last_peered":"2024-09-18T01:04:13.461381+0000","last_clean":"2024-09-18T01:04:13.461381+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:04:13.461381+0000","last_undegraded":"2024-09-18T01:04:13.461381+0000","last_fullsized":"2024-09-18T01:04:13.461381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:03:04.021107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073955169999999999,"stat_sum":{"num_bytes":11604572,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":5883,"num_read_kb":17152,"num_write":575,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.454752+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:04:13.454752+0000","last_peered":"2024-09-18T01:04:13.454752+0000","last_clean":"2024-09-18T01:04:13.454752+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:04:13.454752+0000","last_undegraded":"2024-09-18T01:04:13.454752+0000","last_fullsized":"2024-09-18T01:04:13.454752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"92'702","reported_seq":6845,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.463338+0000","last_change":"2024-09-18T01:04:13.463338+0000","last_active":"2024-09-18T01:04:13.463338+0000","last_peered":"2024-09-18T01:04:13.463338+0000","last_clean":"2024-09-18T01:04:13.463338+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:04:13.463338+0000","last_undegraded":"2024-09-18T01:04:13.463338+0000","last_fullsized":"2024-09-18T01:04:13.463338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:26.341179+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085655820000000004,"stat_sum":{"num_bytes":83316261,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":8003,"num_read_kb":75213,"num_write":948,"num_write_kb":12033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.460454+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:04:13.460454+0000","last_peered":"2024-09-18T01:04:13.460454+0000","last_clean":"2024-09-18T01:04:13.460454+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:04:13.460454+0000","last_undegraded":"2024-09-18T01:04:13.460454+0000","last_fullsized":"2024-09-18T01:04:13.460454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"91'2017","reported_seq":10393,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.465521+0000","last_change":"2024-09-18T01:04:13.465726+0000","last_active":"2024-09-18T01:04:13.465521+0000","last_peered":"2024-09-18T01:04:13.465521+0000","last_clean":"2024-09-18T01:04:13.465521+0000","last_became_active":"2024-09-18T01:03:45.291927+0000","last_became_peered":"2024-09-18T01:03:45.291927+0000","last_unstale":"2024-09-18T01:04:13.465521+0000","last_undegraded":"2024-09-18T01:04:13.465521+0000","last_fullsized":"2024-09-18T01:04:13.465521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:29.207566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004887888,"stat_sum":{"num_bytes":39640881,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":19254,"num_read_kb":46787,"num_write":3291,"num_write_kb":9342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":2222926,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.454472+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:04:13.454472+0000","last_peered":"2024-09-18T01:04:13.454472+0000","last_clean":"2024-09-18T01:04:13.454472+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:04:13.454472+0000","last_undegraded":"2024-09-18T01:04:13.454472+0000","last_fullsized":"2024-09-18T01:04:13.454472+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'444","reported_seq":5815,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.463792+0000","last_change":"2024-09-18T01:04:13.465445+0000","last_active":"2024-09-18T01:04:13.463792+0000","last_peered":"2024-09-18T01:04:13.463792+0000","last_clean":"2024-09-18T01:04:13.463792+0000","last_became_active":"2024-09-18T01:03:45.291124+0000","last_became_peered":"2024-09-18T01:03:45.291124+0000","last_unstale":"2024-09-18T01:04:13.463792+0000","last_undegraded":"2024-09-18T01:04:13.463792+0000","last_fullsized":"2024-09-18T01:04:13.463792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:43:31.014668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004737369,"stat_sum":{"num_bytes":16131156,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":5771,"num_read_kb":21947,"num_write":470,"num_write_kb":7398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":270420,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.460633+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:04:13.460633+0000","last_peered":"2024-09-18T01:04:13.460633+0000","last_clean":"2024-09-18T01:04:13.460633+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:04:13.460633+0000","last_undegraded":"2024-09-18T01:04:13.460633+0000","last_fullsized":"2024-09-18T01:04:13.460633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"92'330","reported_seq":7898,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.529295+0000","last_change":"2024-09-18T01:04:13.459755+0000","last_active":"2024-09-18T01:04:13.529295+0000","last_peered":"2024-09-18T01:04:13.529295+0000","last_clean":"2024-09-18T01:04:13.529295+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:04:13.529295+0000","last_undegraded":"2024-09-18T01:04:13.529295+0000","last_fullsized":"2024-09-18T01:04:13.529295+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:48.535692+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0067530610000000003,"stat_sum":{"num_bytes":24014880,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":7490,"num_read_kb":41288,"num_write":350,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.3","version":"92'899","reported_seq":6928,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.619177+0000","last_change":"2024-09-18T01:04:13.462867+0000","last_active":"2024-09-18T01:04:13.619177+0000","last_peered":"2024-09-18T01:04:13.619177+0000","last_clean":"2024-09-18T01:04:13.619177+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:04:13.619177+0000","last_undegraded":"2024-09-18T01:04:13.619177+0000","last_fullsized":"2024-09-18T01:04:13.619177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:38:49.331400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036055919999999999,"stat_sum":{"num_bytes":48902370,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":9491,"num_read_kb":61307,"num_write":1190,"num_write_kb":7849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.2","version":"18'2","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.460437+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:04:13.460437+0000","last_peered":"2024-09-18T01:04:13.460437+0000","last_clean":"2024-09-18T01:04:13.460437+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:04:13.460437+0000","last_undegraded":"2024-09-18T01:04:13.460437+0000","last_fullsized":"2024-09-18T01:04:13.460437+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"92'272","reported_seq":6463,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.470716+0000","last_change":"2024-09-18T01:04:13.471003+0000","last_active":"2024-09-18T01:04:13.470716+0000","last_peered":"2024-09-18T01:04:13.470716+0000","last_clean":"2024-09-18T01:04:13.470716+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:04:13.470716+0000","last_undegraded":"2024-09-18T01:04:13.470716+0000","last_fullsized":"2024-09-18T01:04:13.470716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:30.709770+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.010935432,"stat_sum":{"num_bytes":53299458,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":6131,"num_read_kb":49422,"num_write":279,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.452941+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:04:13.452941+0000","last_peered":"2024-09-18T01:04:13.452941+0000","last_clean":"2024-09-18T01:04:13.452941+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:04:13.452941+0000","last_undegraded":"2024-09-18T01:04:13.452941+0000","last_fullsized":"2024-09-18T01:04:13.452941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'328","reported_seq":4216,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.454324+0000","last_change":"2024-09-18T01:04:13.455081+0000","last_active":"2024-09-18T01:04:13.454324+0000","last_peered":"2024-09-18T01:04:13.454324+0000","last_clean":"2024-09-18T01:04:13.454324+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:04:13.454324+0000","last_undegraded":"2024-09-18T01:04:13.454324+0000","last_fullsized":"2024-09-18T01:04:13.454324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:31:08.002436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684120000000001,"stat_sum":{"num_bytes":29988352,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":4652,"num_read_kb":30384,"num_write":291,"num_write_kb":9930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.453811+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:04:13.453811+0000","last_peered":"2024-09-18T01:04:13.453811+0000","last_clean":"2024-09-18T01:04:13.453811+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:04:13.453811+0000","last_undegraded":"2024-09-18T01:04:13.453811+0000","last_fullsized":"2024-09-18T01:04:13.453811+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"91'422","reported_seq":6634,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.462799+0000","last_change":"2024-09-18T01:04:13.463003+0000","last_active":"2024-09-18T01:04:13.462799+0000","last_peered":"2024-09-18T01:04:13.462799+0000","last_clean":"2024-09-18T01:04:13.462799+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:04:13.462799+0000","last_undegraded":"2024-09-18T01:04:13.462799+0000","last_fullsized":"2024-09-18T01:04:13.462799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:33.271827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025237749999999998,"stat_sum":{"num_bytes":17028387,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":7216,"num_read_kb":41772,"num_write":392,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"2.3","version":"16'1","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.453361+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:04:13.453361+0000","last_peered":"2024-09-18T01:04:13.453361+0000","last_clean":"2024-09-18T01:04:13.453361+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:04:13.453361+0000","last_undegraded":"2024-09-18T01:04:13.453361+0000","last_fullsized":"2024-09-18T01:04:13.453361+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":319,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.453823+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:04:13.453823+0000","last_peered":"2024-09-18T01:04:13.453823+0000","last_clean":"2024-09-18T01:04:13.453823+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:04:13.453823+0000","last_undegraded":"2024-09-18T01:04:13.453823+0000","last_fullsized":"2024-09-18T01:04:13.453823+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'625","reported_seq":8085,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.453461+0000","last_change":"2024-09-18T01:04:13.453842+0000","last_active":"2024-09-18T01:04:13.453461+0000","last_peered":"2024-09-18T01:04:13.453461+0000","last_clean":"2024-09-18T01:04:13.453461+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:04:13.453461+0000","last_undegraded":"2024-09-18T01:04:13.453461+0000","last_fullsized":"2024-09-18T01:04:13.453461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:28.618365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085762800000000004,"stat_sum":{"num_bytes":41859689,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":8659,"num_read_kb":60969,"num_write":830,"num_write_kb":15561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.9","version":"90'397","reported_seq":4750,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.462369+0000","last_change":"2024-09-18T01:04:13.462952+0000","last_active":"2024-09-18T01:04:13.462369+0000","last_peered":"2024-09-18T01:04:13.462369+0000","last_clean":"2024-09-18T01:04:13.462369+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:04:13.462369+0000","last_undegraded":"2024-09-18T01:04:13.462369+0000","last_fullsized":"2024-09-18T01:04:13.462369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:53:21.290959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036738209999999999,"stat_sum":{"num_bytes":28088920,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":4857,"num_read_kb":30766,"num_write":486,"num_write_kb":10689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.a","version":"92'673","reported_seq":5008,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.460030+0000","last_change":"2024-09-18T01:04:13.460030+0000","last_active":"2024-09-18T01:04:13.460030+0000","last_peered":"2024-09-18T01:04:13.460030+0000","last_clean":"2024-09-18T01:04:13.460030+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:04:13.460030+0000","last_undegraded":"2024-09-18T01:04:13.460030+0000","last_fullsized":"2024-09-18T01:04:13.460030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:21:36.175944+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059071799999999997,"stat_sum":{"num_bytes":29537418,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":6355,"num_read_kb":22541,"num_write":857,"num_write_kb":4897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.b","version":"91'438","reported_seq":6460,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.463119+0000","last_change":"2024-09-18T01:04:13.463721+0000","last_active":"2024-09-18T01:04:13.463119+0000","last_peered":"2024-09-18T01:04:13.463119+0000","last_clean":"2024-09-18T01:04:13.463119+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:04:13.463119+0000","last_undegraded":"2024-09-18T01:04:13.463119+0000","last_fullsized":"2024-09-18T01:04:13.463119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:55.695449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031148090000000001,"stat_sum":{"num_bytes":22155361,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":7899,"num_read_kb":38034,"num_write":379,"num_write_kb":6910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.c","version":"91'319","reported_seq":5704,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.455192+0000","last_change":"2024-09-18T01:04:13.455451+0000","last_active":"2024-09-18T01:04:13.455192+0000","last_peered":"2024-09-18T01:04:13.455192+0000","last_clean":"2024-09-18T01:04:13.455192+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:04:13.455192+0000","last_undegraded":"2024-09-18T01:04:13.455192+0000","last_fullsized":"2024-09-18T01:04:13.455192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:14:14.641091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001146836,"stat_sum":{"num_bytes":26779136,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":6017,"num_read_kb":28902,"num_write":289,"num_write_kb":5471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.d","version":"92'480","reported_seq":5830,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.563001+0000","last_change":"2024-09-18T01:04:13.456562+0000","last_active":"2024-09-18T01:04:13.563001+0000","last_peered":"2024-09-18T01:04:13.563001+0000","last_clean":"2024-09-18T01:04:13.563001+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:04:13.563001+0000","last_undegraded":"2024-09-18T01:04:13.563001+0000","last_fullsized":"2024-09-18T01:04:13.563001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:03.577732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021772179999999999,"stat_sum":{"num_bytes":18480128,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":6457,"num_read_kb":28618,"num_write":506,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.e","version":"91'341","reported_seq":6499,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.459305+0000","last_change":"2024-09-18T01:04:13.459852+0000","last_active":"2024-09-18T01:04:13.459305+0000","last_peered":"2024-09-18T01:04:13.459305+0000","last_clean":"2024-09-18T01:04:13.459305+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:04:13.459305+0000","last_undegraded":"2024-09-18T01:04:13.459305+0000","last_fullsized":"2024-09-18T01:04:13.459305+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:44:36.030185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080554400000000005,"stat_sum":{"num_bytes":44646465,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":6547,"num_read_kb":36692,"num_write":310,"num_write_kb":7776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.f","version":"91'554","reported_seq":12859,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.582926+0000","last_change":"2024-09-18T01:04:13.461295+0000","last_active":"2024-09-18T01:04:13.582926+0000","last_peered":"2024-09-18T01:04:13.582926+0000","last_clean":"2024-09-18T01:04:13.582926+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:04:13.582926+0000","last_undegraded":"2024-09-18T01:04:13.582926+0000","last_fullsized":"2024-09-18T01:04:13.582926+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:33.131871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067058889999999996,"stat_sum":{"num_bytes":27423775,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":10794,"num_read_kb":25755,"num_write":421,"num_write_kb":7503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.10","version":"91'374","reported_seq":4191,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.460296+0000","last_change":"2024-09-18T01:04:13.462779+0000","last_active":"2024-09-18T01:04:13.460296+0000","last_peered":"2024-09-18T01:04:13.460296+0000","last_clean":"2024-09-18T01:04:13.460296+0000","last_became_active":"2024-09-18T01:03:45.291335+0000","last_became_peered":"2024-09-18T01:03:45.291335+0000","last_unstale":"2024-09-18T01:04:13.460296+0000","last_undegraded":"2024-09-18T01:04:13.460296+0000","last_fullsized":"2024-09-18T01:04:13.460296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:34:42.917289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002353922,"stat_sum":{"num_bytes":23320382,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":4632,"num_read_kb":25565,"num_write":388,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2210583,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.11","version":"91'364","reported_seq":4161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.454217+0000","last_change":"2024-09-18T01:04:13.454926+0000","last_active":"2024-09-18T01:04:13.454217+0000","last_peered":"2024-09-18T01:04:13.454217+0000","last_clean":"2024-09-18T01:04:13.454217+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:04:13.454217+0000","last_undegraded":"2024-09-18T01:04:13.454217+0000","last_fullsized":"2024-09-18T01:04:13.454217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:49:20.276052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096354199999999996,"stat_sum":{"num_bytes":25847808,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":4079,"num_read_kb":19488,"num_write":359,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.12","version":"92'894","reported_seq":10095,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.459782+0000","last_change":"2024-09-18T01:04:13.459782+0000","last_active":"2024-09-18T01:04:13.459782+0000","last_peered":"2024-09-18T01:04:13.459782+0000","last_clean":"2024-09-18T01:04:13.459782+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:04:13.459782+0000","last_undegraded":"2024-09-18T01:04:13.459782+0000","last_fullsized":"2024-09-18T01:04:13.459782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:13:52.404244+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0057641719999999997,"stat_sum":{"num_bytes":51846976,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":12555,"num_read_kb":52134,"num_write":1175,"num_write_kb":15840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1e","length":"4"}]},{"pgid":"3.13","version":"91'285","reported_seq":3666,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-18T01:04:13.455482+0000","last_change":"2024-09-18T01:04:13.455482+0000","last_active":"2024-09-18T01:04:13.455482+0000","last_peered":"2024-09-18T01:04:13.455482+0000","last_clean":"2024-09-18T01:04:13.455482+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:04:13.455482+0000","last_undegraded":"2024-09-18T01:04:13.455482+0000","last_fullsized":"2024-09-18T01:04:13.455482+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:30.728373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012219760000000001,"stat_sum":{"num_bytes":20653153,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":3250,"num_read_kb":13082,"num_write":285,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1027106823,"num_objects":1119,"num_object_clones":272,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1119,"num_whiteouts":400,"num_read":258134,"num_read_kb":1194095,"num_write":21560,"num_write_kb":286069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":626,"num_bytes_recovered":29195543,"num_keys_recovered":73,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40103936,"data_stored":309936142,"data_compressed":7743164,"data_compressed_allocated":24854528,"data_compressed_original":297180160,"omap_allocated":0,"internal_metadata":0},"log_size":18579,"ondisk_log_size":18579,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640280,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640282,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67000,"kb_used_data":9444,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304840,"statfs":{"total":96636764160,"available":96568156160,"internally_reserved":0,"allocated":9670656,"data_stored":62629574,"data_compressed":1672151,"data_compressed_allocated":5136384,"data_compressed_original":59345408,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66372,"kb_used_data":8756,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305468,"statfs":{"total":96636764160,"available":96568799232,"internally_reserved":0,"allocated":8966144,"data_stored":55589039,"data_compressed":1439441,"data_compressed_allocated":4349952,"data_compressed_original":52157440,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640280,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78892,"kb_used_data":7448,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94292948,"statfs":{"total":96636764160,"available":96555978752,"internally_reserved":0,"allocated":7626752,"data_stored":57906242,"data_compressed":1189824,"data_compressed_allocated":4337664,"data_compressed_original":55587840,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87684,"kb_used_data":12080,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94284156,"statfs":{"total":96636764160,"available":96546975744,"internally_reserved":0,"allocated":12369920,"data_stored":90301281,"data_compressed":2454348,"data_compressed_allocated":7565312,"data_compressed_original":86858240,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640281,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60232,"kb_used_data":6836,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311608,"statfs":{"total":96636764160,"available":96575086592,"internally_reserved":0,"allocated":7000064,"data_stored":46876520,"data_compressed":1004500,"data_compressed_allocated":3538944,"data_compressed_original":44378112,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5857280,"data_stored":45848985,"data_compressed":995950,"data_compressed_allocated":3502080,"data_compressed_original":43804672,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11284480,"data_stored":89864114,"data_compressed":2454348,"data_compressed_allocated":7565312,"data_compressed_original":86858240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6545408,"data_stored":57469094,"data_compressed":1189824,"data_compressed_allocated":4337664,"data_compressed_original":55587840,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7827456,"data_stored":54561523,"data_compressed":1430891,"data_compressed_allocated":4313088,"data_compressed_original":51584000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8589312,"data_stored":62192426,"data_compressed":1672151,"data_compressed_allocated":5136384,"data_compressed_original":59345408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:04:16.035 INFO:tasks.thrashosds.thrasher:Setting 3.1f to [4, 0, 2, 5] 2024-09-18T01:04:16.035 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1f', '4', '0', '2', '5'] 2024-09-18T01:04:16.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1f 4 0 2 5 2024-09-18T01:04:16.073 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.752565583999996, 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.752545075, 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.752525270000007, 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.752490629999997, 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.752469129000005, 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.752445952000002, 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748457119999998, 2024-09-18T01:04:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748437566999996, 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748417063000005, 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748396198999998, 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748376299, 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748352210999997, 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748320891000006, 2024-09-18T01:04:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748276285000003, 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748245494000003, 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748222009000003, 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748192954000004, 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.748172194999995, 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.490231448000003, 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.490147739000001, 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:16.167 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:16.167 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:16.071+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:16.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:16.169 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:16.169 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:16.169 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:16.170 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:16.170 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:16.170 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:16.178 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:16.179 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:16.179 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:16.178+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:16.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:16.196 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:16.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:16.213 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:16.272 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:16.271+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:16.288 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:16.289 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:16.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:16.373 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:16.372+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:16.405 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:16.405 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:16.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:16.474 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:16.473+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:16.572 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:16.572 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:16.577 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:16.576+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:16.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:16.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:04:16.678 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:16.677+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:16.692 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:16.692 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:16.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:16.781 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:16.780+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:16.810 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:16.810 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:16.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:16.882 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:16.881+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:16.926 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:16.926 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:16.935 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:16.982 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:16.982+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:17.037 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.037 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.046 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:17.084 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:17.083+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:17.149 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.149 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.158 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:17.201 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:17.200+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:17.259 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.260 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.268 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:17.309 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:17.308+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:17.388 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:17.386+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:17.411 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.412 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:17.481 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.1f pg_upmap_items mapping to [4->0,2->5] 2024-09-18T01:04:17.482 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:04:17.490 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:17.488+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:17.576 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.576 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.585 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:17.630 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:17.629+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:17.688 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.688 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.691 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:17.690+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:17.697 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:17.792 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:17.791+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:17.840 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.840 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.849 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:17.933 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:17.932+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:17.991 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:17.991 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:17.993 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:17.992+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:17.999 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:18.094 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:18.093+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:18.142 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:18.142 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:18.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:18.195 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:18.194+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:18.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:18.266 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:18.266 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:18.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:18.296 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:18.295+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:18.362 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:18.384 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:18.397 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:18.396+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:18.402 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:18.421 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:18.437 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:18.437 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:18.438 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:18.438 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:18.438 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:18.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:18.454 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:18.454 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:18.498 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:18.496+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:18.560 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:18.560 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:18.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:18.560 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.239849855000003, 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:18.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:18.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:18.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.239829345999993, 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:18.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.239809541, 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:18.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:18.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.239774901000004, 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:18.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:18.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.239753399999998, 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:18.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:18.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.239730222999995, 2024-09-18T01:04:18.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:18.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235741391000005, 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:18.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235721838000003, 2024-09-18T01:04:18.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:18.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235701333999998, 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:18.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:18.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235680470000005, 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235660569999993, 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235636482000004, 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:18.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235605161999999, 2024-09-18T01:04:18.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:18.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:18.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235560555999996, 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:18.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235529764999995, 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235506279999996, 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:18.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:18.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235477224999997, 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:18.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.235456466000002, 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:18.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:18.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 35.977515719000003, 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:18.644 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:18.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 35.977432010000001, 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:18.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:18.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:18.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:18.655 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:18.656 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:18.655+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:18.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:18.757 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:18.756+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:18.765 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:18.765 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:18.765 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:18.766 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:18.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:18.858 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:18.857+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:18.880 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:18.881 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:18.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:18.959 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:18.958+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 90.718730770999997, 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:18.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:18.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:19.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:19.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 90.714104868000007, 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:19.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:19.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 90.714026818999997, 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:19.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:19.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:19.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.657230776000006, 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:19.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.657204911999997, 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:19.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.657182431999999, 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:19.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.657161396000006, 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:19.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.657134002000006, 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:19.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.657109696999996, 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:19.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653710214, 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653685654, 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:04:19.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653644137000001, 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:04:19.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:19.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653623635000002, 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:19.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653603168999993, 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:19.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653561828999997, 2024-09-18T01:04:19.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:19.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:19.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653541790999995, 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:19.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:19.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653499460000006, 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:19.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653478468000003, 2024-09-18T01:04:19.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:19.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:19.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.653427497999999, 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:19.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:19.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:19.091 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.421287110000002, 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:19.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.098 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:19.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:19.100 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:19.098+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:19.200 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:19.199+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:19.208 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:19.208 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:19.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:19.301 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:19.300+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:19.332 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:19.332 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:19.333 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:19.333 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:19.333 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:19.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:19.402 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:19.401+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:19.503 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:19.502+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:19.509 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.773053564999998, 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.773033607000002, 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.773014665999995, 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.772993157000002, 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.772967429000005, 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764235569999997, 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:04:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:04:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764212856, 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764193726000002, 2024-09-18T01:04:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764174879999999, 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764156001999993, 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764134136999999, 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764110501999994, 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764076852000002, 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764048997000003, 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764029174000001, 2024-09-18T01:04:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.764008739999994, 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.76398666, 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.763964337999994, 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.763941942000002, 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.763921522999993, 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:19.636 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:19.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:04:19.638 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:19.639 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:19.637+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:19.738 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:19.737+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:19.740 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:19.740 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:19.740 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:19.741 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:19.749 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:19.839 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:19.838+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:19.891 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:19.891 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:19.891 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:19.891 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:19.891 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:19.899 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:19.940 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:19.939+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:20.041 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:20.040+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:20.043 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:20.043 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:20.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.998130853999996, 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.044 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:04:20.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:20.048 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.998119703, 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:04:20.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.998107872999995, 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:04:20.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:04:20.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.998091189999997, 2024-09-18T01:04:20.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:04:20.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:04:20.061 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.998079953000001, 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.062 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:20.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.066 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.996757848000001, 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:04:20.070 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.996745133000005, 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.071 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:04:20.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.996700149000006, 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.076 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:04:20.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.996672524999994, 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.080 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:04:20.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.993125145999997, 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:04:20.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:04:20.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:04:20.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.992765039000005, 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:04:20.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:04:20.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:04:20.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.992738926000001, 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.093 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:20.097 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.990559537999999, 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:04:20.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.101 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.990549743000003, 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:04:20.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.990538963999995, 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.106 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.110 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.990519168000006, 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.111 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:04:20.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.115 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.988735648000002, 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.116 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:04:20.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:04:20.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.988708810000006, 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.120 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.121 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 77.988698262, 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:20.125 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:04:20.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 42.516771386000002, 2024-09-18T01:04:20.129 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:04:20.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:04:20.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:20.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.134 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:20.135 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:20.142 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:20.141+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:20.238 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:20.238 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:20.238 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:20.238 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:20.242 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:20.242+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:20.247 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:04:20.357 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:20.357 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:20.357 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:20.357 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:20.357 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:20.358 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:20.357+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:20.366 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:20.443 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:20.442+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:20.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:20.474 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.474 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.176517821000004, 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.475 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:20.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:20.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.176493703999995, 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.479 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:20.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.176414464000004, 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.486 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.176378244999995, 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:20.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.490 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.176349631999997, 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.491 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:20.494 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.176328767000001, 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.495 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:20.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:20.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175711050000004, 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.499 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.502 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175681783000002, 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:20.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175615492000006, 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:20.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.510 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175494724999993, 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:20.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.514 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175276279000002, 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.515 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:20.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:20.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.519 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175254917999993, 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:20.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:20.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175100979000007, 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.524 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:20.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175081261000003, 2024-09-18T01:04:20.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:20.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.532 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175060951000006, 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.536 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175041065000002, 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:20.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175021106000003, 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:20.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.175001108000004, 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:20.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.548 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.174981441, 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.549 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:20.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:20.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:20.553 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 74.174961538999995, 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:20.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:20.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:20.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:20.557 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:20.558 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:20.559 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:20.557+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:20.590 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:20.590 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:20.607 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:20.607 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:20.623 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:20.623 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:20.641 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:20.641 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:20.656 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:20.656 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:20.659 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:20.658+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:20.663 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:20.663 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:20.663 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:20.663 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:20.672 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:20.673 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:20.673 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:20.760 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:20.759+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:20.814 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:20.815 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:20.823 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:20.864 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:20.863+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:20.963 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:20.962+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:20.965 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:20.965 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:20.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:21.064 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:21.063+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:21.083 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:21.083 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:21.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:21.176 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:21.175+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:21.239 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:21.239 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:21.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:21.265 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:21.264+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:21.366 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:21.365+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:21.408 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:21.408 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:21.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:21.467 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:21.466+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:21.528 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:21.528 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:21.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:21.580 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:21.579+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:21.643 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:21.643 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:21.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:21.668 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:21.667+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:21.769 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:21.768+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:21.799 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:21.800 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:21.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:21.893 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:21.892+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:21.956 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:21.956 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:21.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:22.006 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:22.005+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:22.072 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:22.070+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:22.111 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:22.112 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:22.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:22.172 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:22.171+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:22.228 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:22.228 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:22.237 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:22.273 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:22.273+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:22.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:04:22.356 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:22.356 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:22.365 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:22.374 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:22.373+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:22.473 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:22.473 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:22.475 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:22.475+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:22.482 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:22.508 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:04:22.508 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:04:22.509 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T01:04:22.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:04:22.577 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:22.576+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:22.586 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:22.587 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:22.595 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:22.679 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:22.678+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:22.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:22.739 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:22.739 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:22.748 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:22.789 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:22.788+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 101, 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-18T01:02:13.493841+0000", 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-18T01:04:22.295764+0000", 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-18T01:02:24.290620+0000", 2024-09-18T01:04:22.799 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-18T01:02:48.274416+0000", 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:04:22.800 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:24.388064+0000", 2024-09-18T01:04:22.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:22.802 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.803 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:22.804 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.805 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-18T01:04:22.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:22.807 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:30.676199+0000", 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:22.808 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:22.809 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:22.810 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:22.811 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2699999809265137, 2024-09-18T01:04:22.813 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2699999809265137, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-18T01:04:22.814 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:40.390732+0000", 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:22.815 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:22.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "101", 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 33, 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 101, 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:22.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:22.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.7000000476837158, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.7000000476837158, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4563fb16-8745-4e75-b654-1e96403d91b9", 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.8100128173828125, 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:22.821 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-18T01:04:22.822 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-18T01:04:22.823 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-18T01:04:22.825 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/1626166637", 2024-09-18T01:04:22.826 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/1626166637", 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/1626166637", 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/1626166637", 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "634965cb-d444-4721-b34a-afbc49105cab", 2024-09-18T01:04:22.827 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.828 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-18T01:04:22.832 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/2210970259", 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/2210970259", 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/2210970259", 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/2210970259", 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "83443eab-2fa4-4c7e-a459-edb1bb0ed8d0", 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.6881561279296875, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 75, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:22.834 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:22.835 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.836 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.838 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/617133675", 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/617133675", 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/617133675", 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/617133675", 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "5797982a-71c1-4d6a-9604-00fbc6bd16d9", 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:22.839 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 99, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.840 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6808", 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6809", 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6810", 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6811", 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.842 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6814", 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6815", 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6812", 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6813", 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6809/151031464", 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6811/151031464", 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6815/151031464", 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6813/151031464", 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:22.845 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f79e3329-5481-4f33-8d28-6386af186492", 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:22.846 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 74, 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6816", 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6817", 2024-09-18T01:04:22.847 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6818", 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.848 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6819", 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6822", 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6823", 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6820", 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6821", 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6817/1499036965", 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6819/1499036965", 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6823/1499036965", 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6821/1499036965", 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:22.851 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "8c61d2ec-b111-4783-a016-336ed33a0694", 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:22.852 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 20, 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6800", 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6801", 2024-09-18T01:04:22.853 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6802", 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.854 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6803", 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6806", 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6807", 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6804", 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6805", 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6801/2310498524", 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6803/2310498524", 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6807/2310498524", 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6805/2310498524", 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:22.857 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:22.858 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.877700+0000", 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.360930+0000", 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.403205+0000", 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:22.860 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.952123+0000", 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:22.861 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:22.099545+0000", 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.694132+0000", 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:22.862 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [ 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.1f", 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 4, 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 0 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:22.863 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-18T01:04:22.864 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 21, 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 9 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.865 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 21, 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-18T01:04:22.866 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:22.867 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:22.868 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:22.868 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:22.869 INFO:tasks.thrashosds.thrasher:j is {'epoch': 101, 'fsid': '62b46b83-8ef2-466f-b28b-2d97f265422e', 'created': '2024-09-18T01:02:13.493841+0000', 'modified': '2024-09-18T01:04:22.295764+0000', 'last_up_change': '2024-09-18T01:02:24.290620+0000', 'last_in_change': '2024-09-18T01:02:48.274416+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:02:24.388064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:02:30.676199+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:02:40.390732+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4563fb16-8745-4e75-b654-1e96403d91b9', 'up': 1, 'in': 1, 'weight': 0.8100128173828125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 1626166637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 1626166637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 1626166637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 1626166637}]}, 'public_addr': '172.21.15.31:6817/1626166637', 'cluster_addr': '172.21.15.31:6819/1626166637', 'heartbeat_back_addr': '172.21.15.31:6823/1626166637', 'heartbeat_front_addr': '172.21.15.31:6821/1626166637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '634965cb-d444-4721-b34a-afbc49105cab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 2210970259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 2210970259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 2210970259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 2210970259}]}, 'public_addr': '172.21.15.31:6801/2210970259', 'cluster_addr': '172.21.15.31:6803/2210970259', 'heartbeat_back_addr': '172.21.15.31:6807/2210970259', 'heartbeat_front_addr': '172.21.15.31:6805/2210970259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'up': 1, 'in': 1, 'weight': 0.6881561279296875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 617133675}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 617133675}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 617133675}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 617133675}]}, 'public_addr': '172.21.15.31:6809/617133675', 'cluster_addr': '172.21.15.31:6811/617133675', 'heartbeat_back_addr': '172.21.15.31:6815/617133675', 'heartbeat_front_addr': '172.21.15.31:6813/617133675', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6808', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6809', 'nonce': 151031464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6810', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6811', 'nonce': 151031464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6814', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6815', 'nonce': 151031464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6812', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6813', 'nonce': 151031464}]}, 'public_addr': '172.21.15.98:6809/151031464', 'cluster_addr': '172.21.15.98:6811/151031464', 'heartbeat_back_addr': '172.21.15.98:6815/151031464', 'heartbeat_front_addr': '172.21.15.98:6813/151031464', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f79e3329-5481-4f33-8d28-6386af186492', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6816', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6817', 'nonce': 1499036965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6818', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6819', 'nonce': 1499036965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6822', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6823', 'nonce': 1499036965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6820', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6821', 'nonce': 1499036965}]}, 'public_addr': '172.21.15.98:6817/1499036965', 'cluster_addr': '172.21.15.98:6819/1499036965', 'heartbeat_back_addr': '172.21.15.98:6823/1499036965', 'heartbeat_front_addr': '172.21.15.98:6821/1499036965', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8c61d2ec-b111-4783-a016-336ed33a0694', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6800', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6801', 'nonce': 2310498524}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6802', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6803', 'nonce': 2310498524}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6806', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6807', 'nonce': 2310498524}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6804', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6805', 'nonce': 2310498524}]}, 'public_addr': '172.21.15.98:6801/2310498524', 'cluster_addr': '172.21.15.98:6803/2310498524', 'heartbeat_back_addr': '172.21.15.98:6807/2310498524', 'heartbeat_front_addr': '172.21.15.98:6805/2310498524', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.877700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.360930+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.403205+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.952123+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:22.099545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.694132+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 4, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:04:22.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:04:22.870 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:22.870 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:22.871 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:22.871 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:22.871 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:22.872 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:22.876 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:22.877 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:22.880 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:22.879+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:22.891 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:22.892 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:22.892 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:22.892 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:22.892 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:22.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:22.894 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:22.901 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:22.981 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:22.980+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:23.006 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:23.006 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:23.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:23.006 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.708769001999997, 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:23.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:23.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.010 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.708744885000002, 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.011 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:23.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.015 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.708665644999996, 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.019 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.708629426000002, 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:23.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.708600813000004, 2024-09-18T01:04:23.024 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.026 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.708579947999993, 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:23.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:23.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707962230999996, 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.033 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:23.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.037 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707932963999994, 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:23.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707866672999998, 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.042 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:23.045 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707745906, 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.046 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.048 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707527459999994, 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.050 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:23.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.054 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707506099, 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:23.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:23.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707352159999999, 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.059 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:23.063 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707332441999995, 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:23.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707312131999998, 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.068 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:23.071 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707292245999994, 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.072 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:23.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:23.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707272286999995, 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.077 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707252288999996, 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:23.081 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707232622000006, 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.086 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:23.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:23.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 76.707212720000001, 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.090 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.094 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:23.095 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:23.096 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:23.094+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:23.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:04:23.173 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-18T01:04:22.361484+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":806814082,"num_objects":958,"num_object_clones":222,"num_object_copies":1916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":958,"num_whiteouts":372,"num_read":259746,"num_read_kb":1195476,"num_write":21993,"num_write_kb":288552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":645,"num_bytes_recovered":29340986,"num_keys_recovered":73,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19094,"ondisk_log_size":19094,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":359520,"kb_used_data":35524,"kb_used_omap":7,"kb_used_meta":323768,"kb_avail":471499680,"statfs":{"total":483183820800,"available":482815672320,"internally_reserved":0,"allocated":36376576,"data_stored":251059778,"data_compressed":5646961,"data_compressed_allocated":19525632,"data_compressed_original":239888384,"omap_allocated":7973,"internal_metadata":331538651},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-220883128,"num_objects":-165,"num_object_clones":-50,"num_object_copies":-330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":-165,"num_whiteouts":-28,"num_read":1495,"num_read_kb":1280,"num_write":272,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":145424,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000489"},"pg_stats":[{"pgid":"3.1f","version":"93'633","reported_seq":10182,"reported_epoch":98,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-18T01:04:19.364241+0000","last_change":"2024-09-18T01:04:19.309681+0000","last_active":"2024-09-18T01:04:19.364241+0000","last_peered":"2024-09-18T01:04:19.364241+0000","last_clean":"2024-09-18T01:04:16.479839+0000","last_became_active":"2024-09-18T01:04:19.305250+0000","last_became_peered":"2024-09-18T01:04:19.305250+0000","last_unstale":"2024-09-18T01:04:19.364241+0000","last_undegraded":"2024-09-18T01:04:19.364241+0000","last_fullsized":"2024-09-18T01:04:19.290900+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054092,"stat_sum":{"num_bytes":40245776,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":12632,"num_read_kb":58132,"num_write":532,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2777632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3","objects":9},{"shards":"3,4","objects":27}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"3"}]},{"pgid":"3.1e","version":"98'1437","reported_seq":7489,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.362096+0000","last_change":"2024-09-18T01:04:19.305672+0000","last_active":"2024-09-18T01:04:19.362096+0000","last_peered":"2024-09-18T01:04:19.362096+0000","last_clean":"2024-09-18T01:04:19.362096+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:04:19.362096+0000","last_undegraded":"2024-09-18T01:04:19.362096+0000","last_fullsized":"2024-09-18T01:04:19.362096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1437,"log_dups_size":0,"ondisk_log_size":1437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:02.192121+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010938542000000001,"stat_sum":{"num_bytes":9551071,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":12178,"num_read_kb":29178,"num_write":1981,"num_write_kb":8243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.1d","version":"98'399","reported_seq":7114,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.298263+0000","last_change":"2024-09-18T01:04:19.298526+0000","last_active":"2024-09-18T01:04:19.298263+0000","last_peered":"2024-09-18T01:04:19.298263+0000","last_clean":"2024-09-18T01:04:19.298263+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:04:19.298263+0000","last_undegraded":"2024-09-18T01:04:19.298263+0000","last_fullsized":"2024-09-18T01:04:19.298263+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:23:31.216008+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0047483999999999998,"stat_sum":{"num_bytes":17811680,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":7352,"num_read_kb":28949,"num_write":393,"num_write_kb":8542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.1c","version":"97'783","reported_seq":12338,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.321966+0000","last_change":"2024-09-18T01:04:19.298517+0000","last_active":"2024-09-18T01:04:19.321966+0000","last_peered":"2024-09-18T01:04:19.321966+0000","last_clean":"2024-09-18T01:04:19.321966+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:04:19.321966+0000","last_undegraded":"2024-09-18T01:04:19.321966+0000","last_fullsized":"2024-09-18T01:04:19.321966+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:36.699815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045274879999999997,"stat_sum":{"num_bytes":36033128,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":15491,"num_read_kb":43237,"num_write":652,"num_write_kb":8638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.1b","version":"97'588","reported_seq":4302,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.298909+0000","last_change":"2024-09-18T01:04:19.299122+0000","last_active":"2024-09-18T01:04:19.298909+0000","last_peered":"2024-09-18T01:04:19.298909+0000","last_clean":"2024-09-18T01:04:19.298909+0000","last_became_active":"2024-09-18T01:03:44.290192+0000","last_became_peered":"2024-09-18T01:03:44.290192+0000","last_unstale":"2024-09-18T01:04:19.298909+0000","last_undegraded":"2024-09-18T01:04:19.298909+0000","last_fullsized":"2024-09-18T01:04:19.298909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:40:58.826088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048900829999999999,"stat_sum":{"num_bytes":13982340,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":5075,"num_read_kb":19548,"num_write":751,"num_write_kb":5925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":1672836,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.1a","version":"97'226","reported_seq":5684,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.326438+0000","last_change":"2024-09-18T01:04:19.298188+0000","last_active":"2024-09-18T01:04:19.326438+0000","last_peered":"2024-09-18T01:04:19.326438+0000","last_clean":"2024-09-18T01:04:19.326438+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:04:19.326438+0000","last_undegraded":"2024-09-18T01:04:19.326438+0000","last_fullsized":"2024-09-18T01:04:19.326438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:52.294442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043483059999999997,"stat_sum":{"num_bytes":5772016,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":5281,"num_read_kb":19265,"num_write":224,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.19","version":"96'573","reported_seq":8297,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.291471+0000","last_change":"2024-09-18T01:04:19.292004+0000","last_active":"2024-09-18T01:04:19.291471+0000","last_peered":"2024-09-18T01:04:19.291471+0000","last_clean":"2024-09-18T01:04:19.291471+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:04:19.291471+0000","last_undegraded":"2024-09-18T01:04:19.291471+0000","last_fullsized":"2024-09-18T01:04:19.291471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:57.592352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001738662,"stat_sum":{"num_bytes":28103753,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":10598,"num_read_kb":39759,"num_write":550,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.18","version":"97'704","reported_seq":8391,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.294971+0000","last_change":"2024-09-18T01:04:19.295610+0000","last_active":"2024-09-18T01:04:19.294971+0000","last_peered":"2024-09-18T01:04:19.294971+0000","last_clean":"2024-09-18T01:04:19.294971+0000","last_became_active":"2024-09-18T01:03:45.291065+0000","last_became_peered":"2024-09-18T01:03:45.291065+0000","last_unstale":"2024-09-18T01:04:19.294971+0000","last_undegraded":"2024-09-18T01:04:19.294971+0000","last_fullsized":"2024-09-18T01:04:19.294971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:50:37.408216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079274599999999999,"stat_sum":{"num_bytes":51746869,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":9232,"num_read_kb":83000,"num_write":683,"num_write_kb":22273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":11170357,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.17","version":"98'866","reported_seq":8050,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.304470+0000","last_change":"2024-09-18T01:04:19.304651+0000","last_active":"2024-09-18T01:04:19.304470+0000","last_peered":"2024-09-18T01:04:19.304470+0000","last_clean":"2024-09-18T01:04:19.304470+0000","last_became_active":"2024-09-18T01:03:45.291814+0000","last_became_peered":"2024-09-18T01:03:45.291814+0000","last_unstale":"2024-09-18T01:04:19.304470+0000","last_undegraded":"2024-09-18T01:04:19.304470+0000","last_fullsized":"2024-09-18T01:04:19.304470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:00:28.771240+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013257678,"stat_sum":{"num_bytes":32196173,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":10591,"num_read_kb":32355,"num_write":1027,"num_write_kb":6700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1702477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.16","version":"97'332","reported_seq":3646,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.295003+0000","last_change":"2024-09-18T01:04:19.295751+0000","last_active":"2024-09-18T01:04:19.295003+0000","last_peered":"2024-09-18T01:04:19.295003+0000","last_clean":"2024-09-18T01:04:19.295003+0000","last_became_active":"2024-09-18T01:03:38.521858+0000","last_became_peered":"2024-09-18T01:03:38.521858+0000","last_unstale":"2024-09-18T01:04:19.295003+0000","last_undegraded":"2024-09-18T01:04:19.295003+0000","last_fullsized":"2024-09-18T01:04:19.295003+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:00:43.597284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052136170000000003,"stat_sum":{"num_bytes":17342638,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":3176,"num_read_kb":32405,"num_write":389,"num_write_kb":7748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.15","version":"96'564","reported_seq":9683,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.295818+0000","last_change":"2024-09-18T01:04:19.296425+0000","last_active":"2024-09-18T01:04:19.295818+0000","last_peered":"2024-09-18T01:04:19.295818+0000","last_clean":"2024-09-18T01:04:19.295818+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:04:19.295818+0000","last_undegraded":"2024-09-18T01:04:19.295818+0000","last_fullsized":"2024-09-18T01:04:19.295818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:33:50.134000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0058180300000000001,"stat_sum":{"num_bytes":19914373,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":11659,"num_read_kb":41430,"num_write":444,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.14","version":"97'513","reported_seq":4987,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.296582+0000","last_change":"2024-09-18T01:04:19.296927+0000","last_active":"2024-09-18T01:04:19.296582+0000","last_peered":"2024-09-18T01:04:19.296582+0000","last_clean":"2024-09-18T01:04:19.296582+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:04:19.296582+0000","last_undegraded":"2024-09-18T01:04:19.296582+0000","last_fullsized":"2024-09-18T01:04:19.296582+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:19.270332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006229437,"stat_sum":{"num_bytes":11473500,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":5883,"num_read_kb":17152,"num_write":576,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.291069+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:04:19.291069+0000","last_peered":"2024-09-18T01:04:19.291069+0000","last_clean":"2024-09-18T01:04:19.291069+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:04:19.291069+0000","last_undegraded":"2024-09-18T01:04:19.291069+0000","last_fullsized":"2024-09-18T01:04:19.291069+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"97'711","reported_seq":6896,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.292201+0000","last_change":"2024-09-18T01:04:19.292827+0000","last_active":"2024-09-18T01:04:19.292201+0000","last_peered":"2024-09-18T01:04:19.292201+0000","last_clean":"2024-09-18T01:04:19.292201+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:04:19.292201+0000","last_undegraded":"2024-09-18T01:04:19.292201+0000","last_fullsized":"2024-09-18T01:04:19.292201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:57:34.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016983689999999999,"stat_sum":{"num_bytes":79278629,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":8003,"num_read_kb":75213,"num_write":950,"num_write_kb":12033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":179,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.291233+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:04:19.291233+0000","last_peered":"2024-09-18T01:04:19.291233+0000","last_clean":"2024-09-18T01:04:19.291233+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:04:19.291233+0000","last_undegraded":"2024-09-18T01:04:19.291233+0000","last_fullsized":"2024-09-18T01:04:19.291233+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"96'2032","reported_seq":10465,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.293616+0000","last_change":"2024-09-18T01:04:19.294240+0000","last_active":"2024-09-18T01:04:19.293616+0000","last_peered":"2024-09-18T01:04:19.293616+0000","last_clean":"2024-09-18T01:04:19.293616+0000","last_became_active":"2024-09-18T01:03:45.291927+0000","last_became_peered":"2024-09-18T01:03:45.291927+0000","last_unstale":"2024-09-18T01:04:19.293616+0000","last_undegraded":"2024-09-18T01:04:19.293616+0000","last_fullsized":"2024-09-18T01:04:19.293616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2032,"log_dups_size":0,"ondisk_log_size":2032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:49.437324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022634909999999999,"stat_sum":{"num_bytes":27578131,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":19309,"num_read_kb":46831,"num_write":3303,"num_write_kb":9342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":2222926,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":179,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.290852+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:04:19.290852+0000","last_peered":"2024-09-18T01:04:19.290852+0000","last_clean":"2024-09-18T01:04:19.290852+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:04:19.290852+0000","last_undegraded":"2024-09-18T01:04:19.290852+0000","last_fullsized":"2024-09-18T01:04:19.290852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'449","reported_seq":5864,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.293121+0000","last_change":"2024-09-18T01:04:19.293527+0000","last_active":"2024-09-18T01:04:19.293121+0000","last_peered":"2024-09-18T01:04:19.293121+0000","last_clean":"2024-09-18T01:04:19.293121+0000","last_became_active":"2024-09-18T01:03:45.291124+0000","last_became_peered":"2024-09-18T01:03:45.291124+0000","last_unstale":"2024-09-18T01:04:19.293121+0000","last_undegraded":"2024-09-18T01:04:19.293121+0000","last_fullsized":"2024-09-18T01:04:19.293121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:11:09.323189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865011,"stat_sum":{"num_bytes":10953812,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":5775,"num_read_kb":21949,"num_write":473,"num_write_kb":7398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":270420,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":179,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.291792+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:04:19.291792+0000","last_peered":"2024-09-18T01:04:19.291792+0000","last_clean":"2024-09-18T01:04:19.291792+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:04:19.291792+0000","last_undegraded":"2024-09-18T01:04:19.291792+0000","last_fullsized":"2024-09-18T01:04:19.291792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"92'330","reported_seq":7940,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.294464+0000","last_change":"2024-09-18T01:04:19.294921+0000","last_active":"2024-09-18T01:04:19.294464+0000","last_peered":"2024-09-18T01:04:19.294464+0000","last_clean":"2024-09-18T01:04:19.294464+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:04:19.294464+0000","last_undegraded":"2024-09-18T01:04:19.294464+0000","last_fullsized":"2024-09-18T01:04:19.294464+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:08:13.680917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063644400000000001,"stat_sum":{"num_bytes":24014880,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":7490,"num_read_kb":41288,"num_write":350,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.3","version":"97'936","reported_seq":7030,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.298645+0000","last_change":"2024-09-18T01:04:19.298905+0000","last_active":"2024-09-18T01:04:19.298645+0000","last_peered":"2024-09-18T01:04:19.298645+0000","last_clean":"2024-09-18T01:04:19.298645+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:04:19.298645+0000","last_undegraded":"2024-09-18T01:04:19.298645+0000","last_fullsized":"2024-09-18T01:04:19.298645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:00.323884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046728569999999999,"stat_sum":{"num_bytes":42097340,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":9548,"num_read_kb":61358,"num_write":1223,"num_write_kb":7868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.2","version":"18'2","reported_seq":171,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.290788+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:04:19.290788+0000","last_peered":"2024-09-18T01:04:19.290788+0000","last_clean":"2024-09-18T01:04:19.290788+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:04:19.290788+0000","last_undegraded":"2024-09-18T01:04:19.290788+0000","last_fullsized":"2024-09-18T01:04:19.290788+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"96'277","reported_seq":6510,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.292575+0000","last_change":"2024-09-18T01:04:19.293620+0000","last_active":"2024-09-18T01:04:19.292575+0000","last_peered":"2024-09-18T01:04:19.292575+0000","last_clean":"2024-09-18T01:04:19.292575+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:04:19.292575+0000","last_undegraded":"2024-09-18T01:04:19.292575+0000","last_fullsized":"2024-09-18T01:04:19.292575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:53:57.244088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00305106,"stat_sum":{"num_bytes":35942146,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":6131,"num_read_kb":49422,"num_write":282,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":179,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.294261+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:04:19.294261+0000","last_peered":"2024-09-18T01:04:19.294261+0000","last_clean":"2024-09-18T01:04:19.294261+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:04:19.294261+0000","last_undegraded":"2024-09-18T01:04:19.294261+0000","last_fullsized":"2024-09-18T01:04:19.294261+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"97'330","reported_seq":4260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.290658+0000","last_change":"2024-09-18T01:04:19.291203+0000","last_active":"2024-09-18T01:04:19.290658+0000","last_peered":"2024-09-18T01:04:19.290658+0000","last_clean":"2024-09-18T01:04:19.290658+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:04:19.290658+0000","last_undegraded":"2024-09-18T01:04:19.290658+0000","last_fullsized":"2024-09-18T01:04:19.290658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:02.932207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072512300000000004,"stat_sum":{"num_bytes":26646016,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":4652,"num_read_kb":30384,"num_write":293,"num_write_kb":9930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":179,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.290238+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:04:19.290238+0000","last_peered":"2024-09-18T01:04:19.290238+0000","last_clean":"2024-09-18T01:04:19.290238+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:04:19.290238+0000","last_undegraded":"2024-09-18T01:04:19.290238+0000","last_fullsized":"2024-09-18T01:04:19.290238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"97'427","reported_seq":6683,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.292630+0000","last_change":"2024-09-18T01:04:19.293035+0000","last_active":"2024-09-18T01:04:19.292630+0000","last_peered":"2024-09-18T01:04:19.292630+0000","last_clean":"2024-09-18T01:04:19.292630+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:04:19.292630+0000","last_undegraded":"2024-09-18T01:04:19.292630+0000","last_fullsized":"2024-09-18T01:04:19.292630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:49.226286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023871130000000002,"stat_sum":{"num_bytes":8573077,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":7220,"num_read_kb":41774,"num_write":397,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"2.3","version":"16'1","reported_seq":157,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.294358+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:04:19.294358+0000","last_peered":"2024-09-18T01:04:19.294358+0000","last_clean":"2024-09-18T01:04:19.294358+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:04:19.294358+0000","last_undegraded":"2024-09-18T01:04:19.294358+0000","last_fullsized":"2024-09-18T01:04:19.294358+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":331,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.290247+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:04:19.290247+0000","last_peered":"2024-09-18T01:04:19.290247+0000","last_clean":"2024-09-18T01:04:19.290247+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:04:19.290247+0000","last_undegraded":"2024-09-18T01:04:19.290247+0000","last_fullsized":"2024-09-18T01:04:19.290247+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"97'638","reported_seq":8139,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.295662+0000","last_change":"2024-09-18T01:04:19.296226+0000","last_active":"2024-09-18T01:04:19.295662+0000","last_peered":"2024-09-18T01:04:19.295662+0000","last_clean":"2024-09-18T01:04:19.295662+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:04:19.295662+0000","last_undegraded":"2024-09-18T01:04:19.295662+0000","last_fullsized":"2024-09-18T01:04:19.295662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:02:59.178598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197432,"stat_sum":{"num_bytes":26679913,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":8659,"num_read_kb":60969,"num_write":835,"num_write_kb":15561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.9","version":"96'416","reported_seq":4817,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.298579+0000","last_change":"2024-09-18T01:04:19.298853+0000","last_active":"2024-09-18T01:04:19.298579+0000","last_peered":"2024-09-18T01:04:19.298579+0000","last_clean":"2024-09-18T01:04:19.298579+0000","last_became_active":"2024-09-18T01:02:41.457548+0000","last_became_peered":"2024-09-18T01:02:41.457548+0000","last_unstale":"2024-09-18T01:04:19.298579+0000","last_undegraded":"2024-09-18T01:04:19.298579+0000","last_fullsized":"2024-09-18T01:04:19.298579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:13:55.904869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048382479999999999,"stat_sum":{"num_bytes":15702574,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":4891,"num_read_kb":30791,"num_write":503,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.a","version":"98'683","reported_seq":5061,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.296122+0000","last_change":"2024-09-18T01:04:19.296518+0000","last_active":"2024-09-18T01:04:19.296122+0000","last_peered":"2024-09-18T01:04:19.296122+0000","last_clean":"2024-09-18T01:04:19.296122+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:04:19.296122+0000","last_undegraded":"2024-09-18T01:04:19.296122+0000","last_fullsized":"2024-09-18T01:04:19.296122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:33.661444+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0060121430000000002,"stat_sum":{"num_bytes":17642634,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":6360,"num_read_kb":22545,"num_write":861,"num_write_kb":4897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.b","version":"97'440","reported_seq":6516,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.292062+0000","last_change":"2024-09-18T01:04:19.292454+0000","last_active":"2024-09-18T01:04:19.292062+0000","last_peered":"2024-09-18T01:04:19.292062+0000","last_clean":"2024-09-18T01:04:19.292062+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:04:19.292062+0000","last_undegraded":"2024-09-18T01:04:19.292062+0000","last_fullsized":"2024-09-18T01:04:19.292062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:30.144559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019270909999999999,"stat_sum":{"num_bytes":22089760,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":7938,"num_read_kb":38066,"num_write":381,"num_write_kb":6910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.c","version":"96'323","reported_seq":5759,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.291339+0000","last_change":"2024-09-18T01:04:19.291882+0000","last_active":"2024-09-18T01:04:19.291339+0000","last_peered":"2024-09-18T01:04:19.291339+0000","last_clean":"2024-09-18T01:04:19.291339+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:04:19.291339+0000","last_undegraded":"2024-09-18T01:04:19.291339+0000","last_fullsized":"2024-09-18T01:04:19.291339+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:59:23.651984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001142905,"stat_sum":{"num_bytes":25142272,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":6051,"num_read_kb":28931,"num_write":289,"num_write_kb":5471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.d","version":"97'485","reported_seq":5876,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.296440+0000","last_change":"2024-09-18T01:04:19.297018+0000","last_active":"2024-09-18T01:04:19.296440+0000","last_peered":"2024-09-18T01:04:19.296440+0000","last_clean":"2024-09-18T01:04:19.296440+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:04:19.296440+0000","last_undegraded":"2024-09-18T01:04:19.296440+0000","last_fullsized":"2024-09-18T01:04:19.296440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:58.711427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064086229999999996,"stat_sum":{"num_bytes":15363072,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":6457,"num_read_kb":28618,"num_write":507,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.e","version":"98'352","reported_seq":6563,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.297516+0000","last_change":"2024-09-18T01:04:19.297860+0000","last_active":"2024-09-18T01:04:19.297516+0000","last_peered":"2024-09-18T01:04:19.297516+0000","last_clean":"2024-09-18T01:04:19.297516+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:04:19.297516+0000","last_undegraded":"2024-09-18T01:04:19.297516+0000","last_fullsized":"2024-09-18T01:04:19.297516+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:18.032104+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.00409846,"stat_sum":{"num_bytes":34534961,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":6584,"num_read_kb":36722,"num_write":313,"num_write_kb":7776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.f","version":"98'565","reported_seq":12911,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.305184+0000","last_change":"2024-09-18T01:04:19.305404+0000","last_active":"2024-09-18T01:04:19.305184+0000","last_peered":"2024-09-18T01:04:19.305184+0000","last_clean":"2024-09-18T01:04:19.305184+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:04:19.305184+0000","last_undegraded":"2024-09-18T01:04:19.305184+0000","last_fullsized":"2024-09-18T01:04:19.305184+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:02:24.231466+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.014457615,"stat_sum":{"num_bytes":15463439,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":10794,"num_read_kb":25755,"num_write":424,"num_write_kb":7503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.10","version":"97'391","reported_seq":4266,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.291537+0000","last_change":"2024-09-18T01:04:19.292578+0000","last_active":"2024-09-18T01:04:19.291537+0000","last_peered":"2024-09-18T01:04:19.291537+0000","last_clean":"2024-09-18T01:04:19.291537+0000","last_became_active":"2024-09-18T01:03:45.291335+0000","last_became_peered":"2024-09-18T01:03:45.291335+0000","last_unstale":"2024-09-18T01:04:19.291537+0000","last_undegraded":"2024-09-18T01:04:19.291537+0000","last_fullsized":"2024-09-18T01:04:19.291537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:28.890170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022257570000000001,"stat_sum":{"num_bytes":17298696,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":4700,"num_read_kb":25619,"num_write":405,"num_write_kb":7613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2210583,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.11","version":"98'370","reported_seq":4209,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.294538+0000","last_change":"2024-09-18T01:04:19.294930+0000","last_active":"2024-09-18T01:04:19.294538+0000","last_peered":"2024-09-18T01:04:19.294538+0000","last_clean":"2024-09-18T01:04:19.294538+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:04:19.294538+0000","last_undegraded":"2024-09-18T01:04:19.294538+0000","last_fullsized":"2024-09-18T01:04:19.294538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:14:56.653468+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0045430519999999997,"stat_sum":{"num_bytes":14658048,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":4079,"num_read_kb":19488,"num_write":363,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.12","version":"98'912","reported_seq":10159,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.296815+0000","last_change":"2024-09-18T01:04:19.297138+0000","last_active":"2024-09-18T01:04:19.296815+0000","last_peered":"2024-09-18T01:04:19.296815+0000","last_clean":"2024-09-18T01:04:19.296815+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:04:19.296815+0000","last_undegraded":"2024-09-18T01:04:19.296815+0000","last_fullsized":"2024-09-18T01:04:19.296815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:27:56.669505+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0061817390000000003,"stat_sum":{"num_bytes":42557106,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":12586,"num_read_kb":52158,"num_write":1191,"num_write_kb":15845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"6"}]},{"pgid":"3.13","version":"96'289","reported_seq":3714,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-18T01:04:19.290887+0000","last_change":"2024-09-18T01:04:19.291302+0000","last_active":"2024-09-18T01:04:19.290887+0000","last_peered":"2024-09-18T01:04:19.290887+0000","last_clean":"2024-09-18T01:04:19.290887+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:04:19.290887+0000","last_undegraded":"2024-09-18T01:04:19.290887+0000","last_fullsized":"2024-09-18T01:04:19.290887+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:38:04.946993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082059400000000001,"stat_sum":{"num_bytes":19833872,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":3254,"num_read_kb":13084,"num_write":287,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":806223695,"num_objects":954,"num_object_clones":222,"num_object_copies":1908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":954,"num_whiteouts":372,"num_read":259629,"num_read_kb":1195375,"num_write":21832,"num_write_kb":286134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":643,"num_bytes_recovered":29340967,"num_keys_recovered":73,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30539776,"data_stored":244823710,"data_compressed":5675720,"data_compressed_allocated":19513344,"data_compressed_original":236234752,"omap_allocated":0,"internal_metadata":0},"log_size":18974,"ondisk_log_size":18974,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640281,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640283,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65276,"kb_used_data":7712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94306564,"statfs":{"total":96636764160,"available":96569921536,"internally_reserved":0,"allocated":7897088,"data_stored":52064626,"data_compressed":1274759,"data_compressed_allocated":4161536,"data_compressed_original":49516032,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68808,"kb_used_data":7072,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94303032,"statfs":{"total":96636764160,"available":96566304768,"internally_reserved":0,"allocated":7241728,"data_stored":46231979,"data_compressed":1091618,"data_compressed_allocated":3624960,"data_compressed_original":43882496,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640281,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77100,"kb_used_data":5648,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94294740,"statfs":{"total":96636764160,"available":96557813760,"internally_reserved":0,"allocated":5783552,"data_stored":42205706,"data_compressed":804569,"data_compressed_allocated":3141632,"data_compressed_original":40568832,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84800,"kb_used_data":9124,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94287040,"statfs":{"total":96636764160,"available":96549928960,"internally_reserved":0,"allocated":9342976,"data_stored":68766552,"data_compressed":1620385,"data_compressed_allocated":5505024,"data_compressed_original":66199040,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640282,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63536,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308304,"statfs":{"total":96636764160,"available":96571703296,"internally_reserved":0,"allocated":6111232,"data_stored":41790915,"data_compressed":855630,"data_compressed_allocated":3092480,"data_compressed_original":39721984,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4988928,"data_stored":39428772,"data_compressed":898759,"data_compressed_allocated":3117056,"data_compressed_original":37952512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8171520,"data_stored":68287561,"data_compressed":1620385,"data_compressed_allocated":5505024,"data_compressed_original":66199040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4624384,"data_stored":41736149,"data_compressed":804569,"data_compressed_allocated":3141632,"data_compressed_original":40568832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6029312,"data_stored":45172070,"data_compressed":1083068,"data_compressed_allocated":3588096,"data_compressed_original":43309056,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6725632,"data_stored":50199158,"data_compressed":1268939,"data_compressed_allocated":4161536,"data_compressed_original":48205312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:04:23.175 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [1, 0] 2024-09-18T01:04:23.175 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.9', '1', '0'] 2024-09-18T01:04:23.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.9 1 0 2024-09-18T01:04:23.197 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:23.195+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:23.240 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:23.240 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:23.241 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:23.241 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:23.249 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:23.288 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:04:23.334 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:23.333+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:23.398 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:23.397+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:23.400 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:23.400 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:23.400 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:23.401 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:23.401 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:23.410 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:23.499 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:23.498+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 111.179680608, 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.556 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.562 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.18166930300001, 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:23.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:23.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:23.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:23.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:23.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 109.173992351, 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.570 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:23.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:23.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:23.576 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.076134401, 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:23.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:23.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:23.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.220053389, 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.583 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:23.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:23.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:23.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.219684092999998, 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.589 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.594 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.217945248000007, 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.595 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:23.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:23.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.217873358000006, 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.599 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:23.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:23.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.130007348000007, 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.603 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:23.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:23.608 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.123816089000002, 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:23.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.122286403999993, 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.613 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:23.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.618 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.121906714999994, 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.619 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.620 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.621 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:23.622 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.623 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.106408311999999, 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.624 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.625 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.626 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.627 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.628 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.629 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.106201553000005, 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.630 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:23.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.632 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:23.633 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.634 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.105783795999997, 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.637 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:23.639 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.105574558000001, 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.640 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.642 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.643 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.105510624000004, 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.644 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:23.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:23.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:23.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.104762418999996, 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:23.648 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.652 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 95.845823613999997, 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 95.844959996, 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:23.658 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:23.663 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:23.665 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:23.663+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:23.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:23.765 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:23.764+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:23.773 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:23.773 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:23.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:23.866 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:23.865+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:23.888 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:23.888 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:23.888 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:23.888 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:23.889 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:23.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:23.966 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:23.965+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.685672100000005, 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.685651590999996, 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.685631786000002, 2024-09-18T01:04:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.685597146000006, 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.685575645, 2024-09-18T01:04:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.685552467999997, 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681563636000007, 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681544083000006, 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681523579, 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:24.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681502714999993, 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681482814999995, 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681458727000006, 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:24.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:24.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681427407000001, 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:24.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:24.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681382800999998, 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:24.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681352009999998, 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681328524999998, 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:24.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681299469999999, 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:24.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.681278711000004, 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.423337963999998, 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:24.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.423254255000003, 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:24.094 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.099 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:24.101 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:24.099+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:24.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:24.201 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:24.200+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:24.209 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:24.209 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:24.209 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:24.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:24.302 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:24.301+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:24.310 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.9 pg_upmap mapping to [1,0] 2024-09-18T01:04:24.335 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:24.335 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:24.335 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:24.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:24.403 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:24.402+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.219642274999998, 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:24.499 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:24.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.215016371999994, 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:24.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:24.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:24.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:24.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.214938322999998, 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:24.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:24.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:24.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:24.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:24.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:24.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:24.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:24.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.158142280000007, 2024-09-18T01:04:24.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:24.526 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:24.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:24.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:24.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:24.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:24.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:24.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:24.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:24.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:24.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:24.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:24.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:24.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:24.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:24.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:24.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:24.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:24.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:24.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.158116415999999, 2024-09-18T01:04:24.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:24.536 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:24.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:24.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:24.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:24.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:24.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.158093936, 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:24.543 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:24.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.158072899999993, 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:24.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.158045505999993, 2024-09-18T01:04:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.158021200999997, 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154621718000001, 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:04:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154597158000001, 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:04:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:04:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154555641000002, 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:04:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154535139000004, 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154514672999994, 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154473332999999, 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154453294999996, 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154410963999993, 2024-09-18T01:04:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154389972000004, 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.154339002, 2024-09-18T01:04:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.922198614000003, 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:24.623 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:24.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:24.625 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:24.624+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:24.725 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:24.724+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:24.773 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:24.773 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:24.773 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:24.773 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:24.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:24.826 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:24.825+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:24.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:24.930 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:24.930 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:24.930 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:24.929+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:24.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:25.029 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:25.029 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:25.029 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:25.028+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:25.046 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:25.063 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:25.082 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:25.089 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:25.089 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:25.098 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:25.099 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:25.116 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:25.143 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:25.142+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:25.213 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:25.213 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:25.222 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:25.231 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:25.231+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:25.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:04:25.332 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:25.331+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:25.381 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:25.433 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:25.432+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:25.526 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:25.526 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:25.534 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:25.535 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:25.533+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:25.635 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:25.634+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:25.676 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:25.676 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:25.685 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:25.735 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:25.734+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:25.789 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:25.789 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:25.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:25.836 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:25.836+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:25.938 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:25.937+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:25.938 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:25.939 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:25.947 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:26.039 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:26.038+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:26.090 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:26.091 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:26.101 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:26.140 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:26.140+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:26.241 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:26.240+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:26.243 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:26.243 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:26.252 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:04:26.341 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:26.341+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:26.401 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:26.401 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:26.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:26.442 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:26.441+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:26.538 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:26.543 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:26.542+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:26.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:26.644 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:26.643+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:26.665 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:26.665 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:26.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:26.758 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:26.757+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:26.822 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:26.822 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:26.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:26.871 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:26.870+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:26.948 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:26.947+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:26.978 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:26.978 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:26.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:27.049 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:27.048+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:27.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:27.135 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:27.135 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:27.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:27.150 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:27.149+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:27.251 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:27.250+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:27.267 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:27.267 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:27.286 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:27.302 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:27.302 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.028914212999993, 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:27.308 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:27.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:27.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.024288310000003, 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:27.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.024210260999993, 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:27.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:27.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:27.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:27.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:27.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.967414218000002, 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:27.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.337 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.967388353999993, 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:27.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:27.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.967365873999995, 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:27.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.967344838000002, 2024-09-18T01:04:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:27.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:27.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.967317444000003, 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:27.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.967293139000006, 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:27.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1348 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963893655999996, 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050920129000000001, 2024-09-18T01:04:27.362 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1348 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342144+0000", 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342147+0000", 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342154+0000", 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4030000000000001e-06 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342160+0000", 2024-09-18T01:04:27.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393050+0000", 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050889743000000001 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393064+0000", 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4269e-05 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1349 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963869095999996, 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050930470999999998, 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1349 2024-09-18T01:04:27.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342168+0000", 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342169+0000", 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342171+0000", 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342178+0000", 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-18T01:04:27.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342182+0000", 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.702e-06 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393093+0000", 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050910894999999998 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393099+0000", 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963827578999997, 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:04:27.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:27.375 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963807076999998, 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963786611000003, 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:27.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963745270999993, 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:27.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963725233000005, 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:27.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963682902000002, 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:27.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963661909999999, 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:27.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:27.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.963610939999995, 2024-09-18T01:04:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:27.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 44.731470551999998, 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:27.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:27.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:27.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:27.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:27.418 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:27.418 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:27.414+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:27.516 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:27.515+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:27.563 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:27.564 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:27.564 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:27.564 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:27.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:27.657 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:27.656+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:27.717 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:27.716+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:27.719 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:27.719 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:27.719 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:27.719 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:27.720 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:27.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:27.818 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:27.817+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:27.877 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:27.877 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:27.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:27.877 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.140620872, 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:27.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:27.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:27.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.140600914000004, 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.140581972999996, 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.140560464000004, 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.140534736000006, 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:27.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131802876999998, 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:27.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:04:27.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:04:27.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131780163000002, 2024-09-18T01:04:27.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:27.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:27.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131761033000004, 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:27.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:27.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131742187, 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:27.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:27.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131723308999995, 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:27.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131701444000001, 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:27.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:27.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131677808999996, 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:27.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131644159000004, 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:27.953 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:27.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:27.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:27.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131616304000005, 2024-09-18T01:04:27.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:27.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:27.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131596481000003, 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:27.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:27.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131576046999996, 2024-09-18T01:04:27.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:27.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:27.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131553967000002, 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:27.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:27.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:27.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131531644999995, 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:27.985 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131509249000004, 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:27.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:27.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:27.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.131488829999995, 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:27.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:27.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:28.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:28.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:28.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:28.004 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:28.005 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:28.006 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:28.004+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:28.106 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:28.105+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:28.112 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:28.112 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:28.112 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:28.112 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:28.121 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:28.208 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:28.207+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:28.264 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:28.264 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:28.265 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:28.265 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:28.265 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:28.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:28.309 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:28.308+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:28.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.336454097000001, 2024-09-18T01:04:28.382 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:28.383 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.384 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:04:28.385 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.336442946000005, 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:04:28.387 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.389 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:04:28.390 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.391 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.336431116, 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:04:28.392 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:28.393 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.394 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:04:28.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.396 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.336414433000002, 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:04:28.397 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.399 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.400 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.401 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.336403196000006, 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.402 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.403 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:04:28.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.405 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.406 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.335081091000006, 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:04:28.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:04:28.409 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.410 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.335068375999995, 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.412 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:04:28.413 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.414 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.415 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.335023391999997, 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.416 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.417 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:04:28.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:04:28.419 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.420 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.334995767999999, 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.421 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.422 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:28.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.424 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.425 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.331448389000002, 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:04:28.427 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.428 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.429 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.430 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.331088281999996, 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.431 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:28.432 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:04:28.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.434 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.331062169000006, 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.436 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:04:28.438 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:04:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:04:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:04:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.439 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.328882781000004, 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.441 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:28.443 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.444 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:04:28.445 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.328872985999993, 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:04:28.446 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.447 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.448 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:04:28.449 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.450 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.328862207, 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.451 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.452 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:04:28.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:04:28.454 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.328842410999997, 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.456 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.457 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.460 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.461 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:04:28.462 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:28.463 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.327058890999993, 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.464 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.465 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.466 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:04:28.467 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.327032052999996, 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.468 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.469 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.470 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:04:28.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.472 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 86.327021505000005, 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:04:28.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:28.474 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.475 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:04:28.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 50.855094629, 2024-09-18T01:04:28.477 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:28.478 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.479 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.480 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:04:28.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.482 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.483 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.483 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:28.484 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:28.485 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:28.483+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:28.585 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:28.584+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:28.628 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:28.628 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:28.628 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:28.628 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:28.637 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:28.721 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:28.720+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:28.781 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:28.781 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:28.781 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:28.781 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:28.781 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:28.787 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:28.786+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:28.789 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:28.888 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:28.887+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:28.933 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:28.933 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:28.933 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:28.933 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:28.933 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.933 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.635427601999993, 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.935 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.936 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:28.937 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.635403484999998, 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.938 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:28.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.635324245000007, 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:28.942 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:28.944 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.945 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.946 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.635288025999998, 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.947 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.949 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:28.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:28.951 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.635259413, 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.952 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:28.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:28.954 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.635238548000004, 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:28.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.957 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.958 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:28.959 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.634620831000007, 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.960 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.961 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.962 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:28.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:28.964 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.634591564000004, 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.965 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.966 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:28.967 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.968 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.634525272999994, 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.969 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.970 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.971 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:28.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.634404505999996, 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.974 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.975 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.976 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:28.977 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.634186060000005, 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.978 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.979 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:28.980 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.981 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.982 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.634164698999996, 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.983 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:28.984 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.985 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:28.986 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.634010759999995, 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.987 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.989 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.990 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.991 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.633991042000005, 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.992 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.993 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.994 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:28.995 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:28.996 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.633970731999995, 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.997 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:28.998 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:28.999 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.000 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.633950846000005, 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:29.001 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.002 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:29.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.004 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.633930887000005, 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.005 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:29.006 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.007 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:29.008 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.633910889000006, 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:29.010 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:29.012 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:29.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.633891222000003, 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.015 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.016 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.017 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:29.018 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 82.633871319999997, 2024-09-18T01:04:29.019 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.020 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.021 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.022 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.023 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:29.025 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:29.025 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:29.023+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:29.125 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:29.124+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:29.172 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:29.173 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:29.173 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:29.173 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:29.181 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:29.265 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:29.264+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:29.324 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:29.324 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:29.325 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:29.325 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:29.325 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:29.326 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:29.325+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:29.333 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:29.333 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:04:29.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:04:29.334 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-18T01:04:29.334 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T01:04:29.334 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T01:04:29.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-18T01:04:29.345 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:04:29.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:29.427 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:29.425+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:29.478 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:29.478 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "" 2024-09-18T01:04:29.478 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:29.492 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 117.112670896, 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.493 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:29.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:29.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:29.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 116.11465959100001, 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:29.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:29.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:29.506 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 115.10698263899999, 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:29.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:29.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:29.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 106.009124689, 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:29.513 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:29.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.153043677, 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.520 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:29.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.152674381, 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.526 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:29.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:29.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:29.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.15093553600001, 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:29.532 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:29.536 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.150863646, 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:29.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.540 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.06299763600001, 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:29.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.056806377, 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:29.547 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:29.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.05527669200001, 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.552 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:29.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:29.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:29.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.05489700299999, 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:29.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:29.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:29.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.0393986, 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:29.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:29.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:29.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.569 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.039191841, 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:29.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:29.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:29.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.038774084, 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:29.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:29.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:29.579 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.038564846, 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.038500912, 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.584 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:29.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.588 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 102.037752707, 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:29.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:29.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.778813902, 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.593 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:29.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:29.599 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.777950284, 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:29.600 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:29.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:29.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:29.605 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:29.607 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:29.608 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:29.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:29.609 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:29.609 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:29.609 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:29.610 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:29.610 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:29.610 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:29.605+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:29.620 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:29.638 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:29.638 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:29.708 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:29.706+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:29.728 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:29.728 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:29.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:29.808 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:29.807+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:29.844 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:29.844 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:29.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:29.909 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:29.908+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:30.001 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.001 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.010 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:30.008+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:30.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:30.110 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:30.109+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:30.117 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.117 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:30.211 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:30.210+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:30.273 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.274 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:30.314 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:30.312+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:30.415 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:30.414+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:30.440 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.440 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:30.519 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:30.518+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:30.598 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.598 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:30.651 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:30.650+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:30.714 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.714 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.722 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:30.721+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:30.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:30.823 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:30.822+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:30.870 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.871 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.879 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:30.926 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:30.925+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:30.988 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:30.988 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:30.997 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:31.038 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:31.037+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:31.130 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:31.130+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:31.139 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:31.139 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:31.148 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:31.231 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:31.230+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:31.251 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:31.251 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:31.259 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:31.334 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:31.333+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:31.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:04:31.409 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:31.410 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:31.419 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:31.435 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:31.434+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:31.525 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:31.525 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:31.534 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:31.537 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:31.535+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:31.635 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:31.635 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:31.638 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:31.636+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:31.644 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:31.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:31.739 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:31.738+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:31.746 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:31.746 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:31.755 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:31.772 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:31.772 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:31.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:31.789 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:31.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:31.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:31.822 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:31.822 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:31.839 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:31.839 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:31.838+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:31.855 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:31.898 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:31.898 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:31.898 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:31.898 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:31.899 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:31.907 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:31.948 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:31.947+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:32.041 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:32.040+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:32.052 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:32.052 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:32.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:32.052 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:32.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.052 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:32.052 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 119.67573037699999, 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:32.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:32.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.677719072, 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:32.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.061 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:32.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.066 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 117.67004212000001, 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:32.067 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:32.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:32.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.074 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 108.57218417, 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:32.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:32.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:32.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:32.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.079 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.079 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:32.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:32.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:32.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:32.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:32.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:32.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:32.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:32.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:32.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:32.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:32.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:32.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:32.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.716103158, 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.085 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:32.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:32.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:32.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:32.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:32.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:32.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:32.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:32.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:32.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:32.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:32.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:32.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:32.094 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.094 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.095 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.095 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.095 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:32.095 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.71573386199999, 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:32.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:32.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:32.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:32.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:32.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:32.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.102 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.102 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:32.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:32.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:32.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:32.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:32.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:32.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.713995017, 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.105 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.106 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:32.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.713923127, 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.110 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:32.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.115 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.626057117, 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.116 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.117 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:32.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:32.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:32.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.122 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.619865858, 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:32.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:32.125 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.618336173, 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.128 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:32.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.134 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.617956484, 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:32.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:32.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:32.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:32.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:32.141 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.60245808099999, 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.142 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.143 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:32.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:32.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:32.152 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:32.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.602251322, 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:32.155 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.156 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:32.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.160 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.161 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.60183356500001, 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.162 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:32.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:32.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:32.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:32.167 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.601624327, 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:32.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:32.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.171 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.601560393, 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.173 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.174 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.60081218800001, 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.178 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:32.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:32.181 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:32.182 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.34187338300001, 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.183 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.184 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:32.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:32.186 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.187 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.188 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 104.341009765, 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.190 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:32.191 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:32.192 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:32.193 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:32.194 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:32.195 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:32.196 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:32.198 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:32.197+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:32.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:32.298 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:32.297+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:32.321 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:32.322 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:32.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:32.354 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:04:32.398 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:32.397+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:32.477 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:32.477 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:32.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:32.530 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:32.529+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.274847390000005, 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.274826880999996, 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.274807076000002, 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.274772436000006, 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.274750935, 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.274727757999997, 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270738926000007, 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270719373000006, 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270698869, 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270678004999994, 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270658104999995, 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270634017000006, 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270602697000001, 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270558090999998, 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270527299999998, 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270503814999998, 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270474759999999, 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.270454001000004, 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.012513253999998, 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.012429545000003, 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:32.689 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:32.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:32.698 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:32.690+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:32.792 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:32.791+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:32.805 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:32.805 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:32.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:32.893 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:32.892+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:32.920 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:32.920 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:32.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:32.994 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:32.993+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.758773566, 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:33.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:33.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:33.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.754147663, 2024-09-18T01:04:33.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:33.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:33.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:33.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:33.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.754069614, 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:33.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:33.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.697273570999997, 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:33.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.697247707000002, 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:33.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:33.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.697225227000004, 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:33.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.697204190999997, 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:33.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:33.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.697176796999997, 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:33.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:33.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.697152492000001, 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:33.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1351 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693686932000006, 2024-09-18T01:04:33.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050931509, 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1351 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:33.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342210+0000", 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342213+0000", 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1830000000000001e-06 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342219+0000", 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342224+0000", 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-18T01:04:33.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393132+0000", 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050907453999999998 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393141+0000", 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9259999999999995e-06 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693666429999993, 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:33.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693645963999998, 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:33.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693604624000002, 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:33.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:33.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693584586, 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:33.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693542254999997, 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:33.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693521262999994, 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:33.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.693470293000004, 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:33.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:33.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.461329904999999, 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:33.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:33.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:24948 3.9 3:945f9464:::rbd_data.1103102c66ed.000000000000015d:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5876622749999996, 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89682947899999998, 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 24948 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448239+0000", 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1579999999999998e-06 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448250+0000", 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0061000000000001e-05 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448256+0000", 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6699999999999997e-06 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448310+0000", 2024-09-18T01:04:33.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3888999999999999e-05 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448321+0000", 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1245e-05 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.342977+0000", 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89465616699999995 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343091+0000", 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011382099999999999 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343168+0000", 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7015999999999996e-05 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343923+0000", 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075445099999999995 2024-09-18T01:04:33.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344936+0000", 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010128820000000001 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345011+0000", 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5519999999999995e-05 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345064+0000", 2024-09-18T01:04:33.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2913e-05 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:24951 3.9 3:90848b56:::rbd_data.1103102c66ed.00000000000001c1:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5874865170000003, 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89740894000000004, 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 24951 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448411+0000", 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6400000000000002e-07 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448415+0000", 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-06 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448422+0000", 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448427+0000", 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.313e-06 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448481+0000", 2024-09-18T01:04:33.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4604000000000001e-05 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448492+0000", 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344058+0000", 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89556604799999995 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344133+0000", 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000006e-05 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344202+0000", 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589000000000001e-05 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345247+0000", 2024-09-18T01:04:33.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010448110000000001 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345693+0000", 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000446203 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345754+0000", 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0445999999999997e-05 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345819+0000", 2024-09-18T01:04:33.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5568999999999995e-05 2024-09-18T01:04:33.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.146 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:33.147 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:33.146+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:33.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:33.248 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:33.247+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:33.255 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:33.256 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:33.256 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:33.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:33.349 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:33.348+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:33.418 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:33.419 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:33.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:33.451 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:33.450+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:33.537 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.801108014999997, 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:33.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:33.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.544 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.801088057000001, 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:33.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:33.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.801069115999994, 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:33.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.801047607000001, 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:33.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.801021879000004, 2024-09-18T01:04:33.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:33.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:33.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22227 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792290019999996, 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066009655, 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22227 2024-09-18T01:04:33.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743064+0000", 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743065+0000", 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743068+0000", 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.067e-06 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743075+0000", 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743080+0000", 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802625+0000", 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059545272000000003 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.802717+0000", 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1592e-05 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803106+0000", 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038906100000000001 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809011+0000", 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059056259999999998 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809022+0000", 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1235e-05 2024-09-18T01:04:33.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809050+0000", 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7912e-05 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809074+0000", 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3487000000000001e-05 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792267305999999, 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:33.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:33.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:33.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792248176000001, 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:33.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:33.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792229329999998, 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:33.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:33.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:33.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792210452000006, 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:33.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:33.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:33.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:33.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792188586999998, 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:33.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:33.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:33.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792164951999993, 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:33.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:33.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:33.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792131302000001, 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:33.615 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:33.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:33.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792103447000002, 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:33.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:33.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:33.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792083624, 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792063189999993, 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:33.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:33.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.79204111, 2024-09-18T01:04:33.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:33.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:33.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:33.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.792018788000007, 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:33.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.791996392000001, 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:33.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:33.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:33.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:33.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.791975973000007, 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:33.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:33.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:33.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:33.668 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:33.669 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:33.670 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:33.668+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:33.770 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:33.769+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:33.772 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:33.772 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:33.772 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:33.781 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:33.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:33.871 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:33.869+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:33.924 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:33.924 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:33.924 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:33.932 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:33.971 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:33.970+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:33.992 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:33.993 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:34.009 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:34.010 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:34.026 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:34.026 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:34.043 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:34.043 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:34.059 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:34.059 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:34.072 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:34.070+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:34.075 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:34.076 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:34.076 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:34.076 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:34.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:34.173 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:34.173+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:34.228 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:34.228 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:34.236 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:34.280 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:34.279+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:34.339 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:34.339 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:34.348 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:34.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:04:34.389 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:34.388+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:34.476 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:34.475+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:34.501 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:34.501 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:34.510 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:34.577 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:34.576+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:34.608 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:04:34.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:04:34.613 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:34.613 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:34.622 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:34.707 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:34.706+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:34.766 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:34.766 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:34.775 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:34.816 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:34.815+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:34.879 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:34.878+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:34.917 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:34.917 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:34.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:34.980 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:34.979+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:35.034 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:35.034 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:35.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:35.084 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:35.083+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:35.182 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:35.181+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:35.190 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:35.190 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:35.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:35.283 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:35.282+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:35.306 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:35.306 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:35.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:35.370 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:04:35.370 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:04:35.384 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:35.383+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:35.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:35.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:35.525 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:35.524+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:35.586 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:35.584+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:35.592 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:35.592 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:35.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:35.687 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:35.686+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:35.750 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:35.750 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:35.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:35.788 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:35.787+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:35.889 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:35.888+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:35.908 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:35.908 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:35.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:35.990 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:35.989+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:36.025 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:36.025 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:36.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:36.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:36.090 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:36.089+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:36.142 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:36.142 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:36.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:36.191 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:36.190+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:36.211 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:36.212 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:36.228 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:36.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:36.257 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:36.257 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:36.260 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:36.261 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:36.266 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:36.278 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:36.292 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:36.291+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:36.294 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:36.294 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:36.366 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:04:36.369 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:36.369 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:36.369 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:36.369 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:36.370 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:36.380 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:36.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:04:36.423 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:36.422+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:36.484 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:36.484 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:36.484 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:36.484 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:36.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.484 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.438265909999998, 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.485 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.486 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.487 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:04:36.488 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.438254759000003, 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.489 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.490 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:04:36.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.438242928999998, 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:04:36.493 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.494 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.497 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.438226245999999, 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:04:36.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:04:36.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.438215009000004, 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:36.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.436892904000004, 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.507 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:36.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.436880188999993, 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.511 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:04:36.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.436835204999994, 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.516 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.436807580999997, 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:04:36.520 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:04:36.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.524 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.433260202, 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:04:36.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:04:36.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:04:36.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.529 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.432900094999994, 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:04:36.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:04:36.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.432873982000004, 2024-09-18T01:04:36.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:36.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:04:36.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.430694594000002, 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:04:36.539 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:04:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.543 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.430684799000005, 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.544 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.430674019999998, 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.548 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:36.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:04:36.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:04:36.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.552 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.430654223999994, 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.553 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23952 3.f 3:f01a3131:::rbd_data.1103ca939bf9.0000000000000001:head [read 29696~14336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.428870704000005, 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.03792185, 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.557 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23952 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053012+0000", 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3799999999999998e-07 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053014+0000", 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6190000000000001e-06 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053017+0000", 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2179999999999999e-06 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053020+0000", 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6139999999999999e-06 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090754+0000", 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037734516000000003 2024-09-18T01:04:36.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090820+0000", 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6391999999999997e-05 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090934+0000", 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011315300000000001 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.428843865999994, 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.562 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:04:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 94.428833318000002, 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:04:36.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:04:36.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:04:36.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:36.571 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 58.956906441999998, 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:36.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:04:36.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.576 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:36.577 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:36.577 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:36.578 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:36.579 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:36.577+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:36.679 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:36.678+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:36.722 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:36.722 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:36.722 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:36.722 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:36.731 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:36.815 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:36.814+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:36.873 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:36.873 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:36.873 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:36.873 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:36.873 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:36.881 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:36.880+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:36.882 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:36.981 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:36.981+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:37.025 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:37.025 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:37.025 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:37.025 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.727810077000001, 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.026 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.027 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:37.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.029 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.727785960000006, 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.030 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.031 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:37.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.033 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.034 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.72770672, 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.035 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.036 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:37.037 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.038 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.727670501000006, 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.039 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:37.040 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:37.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.042 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:37.043 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.727641887999994, 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.044 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:37.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.046 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.047 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.727621022999998, 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.048 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:37.050 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.051 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.727003306, 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.052 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.053 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.054 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:37.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.056 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726974038999998, 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.057 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.058 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.060 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726907748000002, 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.061 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.062 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:37.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.064 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:37.065 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726786981000004, 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.066 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.067 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:37.068 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.069 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726568534999998, 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.070 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.071 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.072 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:37.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726547174000004, 2024-09-18T01:04:37.074 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.076 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.077 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.078 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726393235000003, 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.079 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.080 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:37.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726373516999999, 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.083 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:37.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:37.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:37.087 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726353207000002, 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.091 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726333320999998, 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:37.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:37.095 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726313361999999, 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:37.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726293364, 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:37.100 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.102 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:37.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.104 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726273696999996, 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.105 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:37.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:37.109 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 90.726253795000005, 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.113 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.114 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.114 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.114 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:37.115 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:37.116 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:37.114+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:37.216 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:37.215+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:37.258 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:37.259 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:37.259 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:37.259 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:37.267 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:37.317 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:37.316+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:37.368 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:04:37.370 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:37.370 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:37.370 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:37.381 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:37.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:04:37.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:04:37.439 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:37.438+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:37.493 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:37.493 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:37.493 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:37.493 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 125.11686458, 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:37.494 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.496 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.497 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:37.498 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.499 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:37.500 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 124.11885327500001, 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.501 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:37.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:37.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:37.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:37.508 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 123.111176323, 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:37.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:37.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.515 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 114.013318373, 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.516 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:37.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:37.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.522 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.157237361, 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.523 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:37.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:37.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.156868065, 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.529 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:37.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:37.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.15512922000001, 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.535 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.539 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.15505733000001, 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:37.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:37.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.06719132000001, 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.544 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:37.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:37.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:37.550 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.061000061, 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:37.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.05947037599999, 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.555 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:37.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.561 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.05909068699999, 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:37.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.567 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.043592284, 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.568 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:37.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:37.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.573 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.04338552500001, 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.574 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:37.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:37.579 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.042967768, 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:37.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:37.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.04275853, 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:37.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:37.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:37.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.042694596, 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:37.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.591 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:37.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.594 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 110.041946391, 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:37.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:37.599 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 109.783007586, 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.600 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:37.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:37.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.605 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 109.782143968, 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:37.606 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:37.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:37.611 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.612 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.612 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.612 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:37.612 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:37.612 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:37.614 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:37.612+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:37.614 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:37.714 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:37.713+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:37.740 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:37.741 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:37.741 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:37.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:37.814 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:37.813+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:37.872 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:37.872 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:37.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:37.916 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:37.915+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:37.996 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:37.996 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:37.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:37.996 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:37.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:37.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:37.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.675554982999998, 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:37.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:37.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:37.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.675534474000003, 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:38.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.675514668999995, 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:38.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.675480028999999, 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:38.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.675458527999993, 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.675435351000004, 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:38.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:38.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:38.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671446519, 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:38.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671426965999999, 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:38.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671406461999993, 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:38.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:38.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671385598000001, 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:38.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.044 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671365698000002, 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:38.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671341609999999, 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:38.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671310289999994, 2024-09-18T01:04:38.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:38.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671265684000005, 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671234893000005, 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:38.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671211408000005, 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:38.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671182353000006, 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:38.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.671161593999997, 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:38.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.082 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.413220846999998, 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:38.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:38.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:38.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.413137138000003, 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:38.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:38.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:38.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:38.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:38.094 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:38.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:38.096 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:38.094+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:38.196 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:38.195+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:38.246 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:38.246 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:38.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:38.286 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:04:38.291 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:04:38.297 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:38.296+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:38.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:38.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:04:38.384 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:38.385 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:38.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:38.398 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:38.397+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:38.446 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:38.446 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:38.467 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:38.468 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:38.499 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:38.498+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:38.501 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:38.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:38.520 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:38.537 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:38.569 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:38.570 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:38.580 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:38.581 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:38.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:38.631 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:38.629+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:38.701 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:38.700+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:38.739 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:38.739 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:38.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:38.802 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:38.800+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:38.855 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:38.855 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:38.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:38.905 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:38.903+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:39.004 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:39.003+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:39.011 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.011 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.020 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:39.105 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:39.104+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:39.134 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.134 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.143 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:39.206 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:39.204+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:39.287 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.287 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.291 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:04:39.297 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:39.319 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:04:39.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:04:39.341 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:39.340+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:39.406 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.406 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.407 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:39.405+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:39.415 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:39.508 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:39.506+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:39.559 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.559 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.567 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:39.609 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:39.607+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:39.670 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:39.671 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:04:39.683 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-18T01:04:39.295882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209103358,"num_objects":389,"num_object_clones":23,"num_object_copies":778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":194,"num_read":263884,"num_read_kb":1199003,"num_write":22653,"num_write_kb":288724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":795,"num_bytes_recovered":31931736,"num_keys_recovered":73,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20209,"ondisk_log_size":20209,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":309352,"kb_used_data":10408,"kb_used_omap":7,"kb_used_meta":298808,"kb_avail":471549848,"statfs":{"total":483183820800,"available":482867044352,"internally_reserved":0,"allocated":10657792,"data_stored":10143447,"data_compressed":159380,"data_compressed_allocated":712704,"data_compressed_original":5264384,"omap_allocated":7973,"internal_metadata":305979611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.930137"},"pg_stats":[{"pgid":"3.1f","version":"110'647","reported_seq":10459,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370731+0000","last_change":"2024-09-18T01:04:34.370950+0000","last_active":"2024-09-18T01:04:34.370731+0000","last_peered":"2024-09-18T01:04:34.370731+0000","last_clean":"2024-09-18T01:04:34.370731+0000","last_became_active":"2024-09-18T01:04:21.303421+0000","last_became_peered":"2024-09-18T01:04:21.303421+0000","last_unstale":"2024-09-18T01:04:34.370731+0000","last_undegraded":"2024-09-18T01:04:34.370731+0000","last_fullsized":"2024-09-18T01:04:34.370731+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:29:31.129611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011175670000000001,"stat_sum":{"num_bytes":9754128,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":14,"num_read":12632,"num_read_kb":58132,"num_write":536,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1e","version":"111'1472","reported_seq":7646,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365748+0000","last_change":"2024-09-18T01:04:34.366154+0000","last_active":"2024-09-18T01:04:34.365748+0000","last_peered":"2024-09-18T01:04:34.365748+0000","last_clean":"2024-09-18T01:04:34.365748+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:04:34.365748+0000","last_undegraded":"2024-09-18T01:04:34.365748+0000","last_fullsized":"2024-09-18T01:04:34.365748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1472,"log_dups_size":0,"ondisk_log_size":1472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:48.922070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064141899999999995,"stat_sum":{"num_bytes":89088,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2011,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1d","version":"112'436","reported_seq":7343,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366703+0000","last_change":"2024-09-18T01:04:34.367041+0000","last_active":"2024-09-18T01:04:34.366703+0000","last_peered":"2024-09-18T01:04:34.366703+0000","last_clean":"2024-09-18T01:04:34.366703+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:04:34.366703+0000","last_undegraded":"2024-09-18T01:04:34.366703+0000","last_fullsized":"2024-09-18T01:04:34.366703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:55.314450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094235900000000004,"stat_sum":{"num_bytes":3716283,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":7596,"num_read_kb":29158,"num_write":416,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1c","version":"112'834","reported_seq":12561,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366174+0000","last_change":"2024-09-18T01:04:34.366358+0000","last_active":"2024-09-18T01:04:34.366174+0000","last_peered":"2024-09-18T01:04:34.366174+0000","last_clean":"2024-09-18T01:04:34.366174+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:04:34.366174+0000","last_undegraded":"2024-09-18T01:04:34.366174+0000","last_fullsized":"2024-09-18T01:04:34.366174+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:26:26.424573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041547200000000001,"stat_sum":{"num_bytes":11531817,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":15622,"num_read_kb":43359,"num_write":701,"num_write_kb":8643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1b","version":"108'621","reported_seq":4435,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366461+0000","last_change":"2024-09-18T01:04:34.366650+0000","last_active":"2024-09-18T01:04:34.366461+0000","last_peered":"2024-09-18T01:04:34.366461+0000","last_clean":"2024-09-18T01:04:34.366461+0000","last_became_active":"2024-09-18T01:03:44.290192+0000","last_became_peered":"2024-09-18T01:03:44.290192+0000","last_unstale":"2024-09-18T01:04:34.366461+0000","last_undegraded":"2024-09-18T01:04:34.366461+0000","last_fullsized":"2024-09-18T01:04:34.366461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:24.341556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073620999999999995,"stat_sum":{"num_bytes":8620129,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":5106,"num_read_kb":19572,"num_write":767,"num_write_kb":5929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":1672836,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1a","version":"109'232","reported_seq":5907,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366121+0000","last_change":"2024-09-18T01:04:34.366344+0000","last_active":"2024-09-18T01:04:34.366121+0000","last_peered":"2024-09-18T01:04:34.366121+0000","last_clean":"2024-09-18T01:04:34.366121+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:04:34.366121+0000","last_undegraded":"2024-09-18T01:04:34.366121+0000","last_fullsized":"2024-09-18T01:04:34.366121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:28.557561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042838899999999999,"stat_sum":{"num_bytes":3674643,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":5403,"num_read_kb":19381,"num_write":228,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.19","version":"111'622","reported_seq":8531,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288960+0000","last_change":"2024-09-18T01:04:33.289281+0000","last_active":"2024-09-18T01:04:33.288960+0000","last_peered":"2024-09-18T01:04:33.288960+0000","last_clean":"2024-09-18T01:04:33.288960+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:04:33.288960+0000","last_undegraded":"2024-09-18T01:04:33.288960+0000","last_fullsized":"2024-09-18T01:04:33.288960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:22.983717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116394,"stat_sum":{"num_bytes":238633,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":10840,"num_read_kb":39965,"num_write":576,"num_write_kb":9910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.18","version":"112'734","reported_seq":8525,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366247+0000","last_change":"2024-09-18T01:04:34.367003+0000","last_active":"2024-09-18T01:04:34.366247+0000","last_peered":"2024-09-18T01:04:34.366247+0000","last_clean":"2024-09-18T01:04:34.366247+0000","last_became_active":"2024-09-18T01:03:45.291065+0000","last_became_peered":"2024-09-18T01:03:45.291065+0000","last_unstale":"2024-09-18T01:04:34.366247+0000","last_undegraded":"2024-09-18T01:04:34.366247+0000","last_fullsized":"2024-09-18T01:04:34.366247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:47.303745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013472930000000001,"stat_sum":{"num_bytes":3448832,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":9266,"num_read_kb":83025,"num_write":701,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":11170357,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.17","version":"111'893","reported_seq":8180,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366865+0000","last_change":"2024-09-18T01:04:34.367318+0000","last_active":"2024-09-18T01:04:34.366865+0000","last_peered":"2024-09-18T01:04:34.366865+0000","last_clean":"2024-09-18T01:04:34.366865+0000","last_became_active":"2024-09-18T01:03:45.291814+0000","last_became_peered":"2024-09-18T01:03:45.291814+0000","last_unstale":"2024-09-18T01:04:34.366865+0000","last_undegraded":"2024-09-18T01:04:34.366865+0000","last_fullsized":"2024-09-18T01:04:34.366865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:42.308373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099354200000000004,"stat_sum":{"num_bytes":16832548,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":10622,"num_read_kb":32379,"num_write":1044,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1702477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.16","version":"112'357","reported_seq":3775,"reported_epoch":113,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-18T01:04:34.369979+0000","last_change":"2024-09-18T01:04:34.369963+0000","last_active":"2024-09-18T01:04:34.369979+0000","last_peered":"2024-09-18T01:04:34.369979+0000","last_clean":"2024-09-18T01:04:34.369979+0000","last_became_active":"2024-09-18T01:03:38.521858+0000","last_became_peered":"2024-09-18T01:03:38.521858+0000","last_unstale":"2024-09-18T01:04:34.369979+0000","last_undegraded":"2024-09-18T01:04:34.369979+0000","last_fullsized":"2024-09-18T01:04:34.369979+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:43.719181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055879699999999998,"stat_sum":{"num_bytes":4040289,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":3213,"num_read_kb":32432,"num_write":407,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.15","version":"111'580","reported_seq":9797,"reported_epoch":113,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-18T01:04:34.369785+0000","last_change":"2024-09-18T01:04:34.369768+0000","last_active":"2024-09-18T01:04:34.369785+0000","last_peered":"2024-09-18T01:04:34.369785+0000","last_clean":"2024-09-18T01:04:34.369785+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:04:34.369785+0000","last_undegraded":"2024-09-18T01:04:34.369785+0000","last_fullsized":"2024-09-18T01:04:34.369785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:39:32.525607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015372529999999999,"stat_sum":{"num_bytes":9430107,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":11665,"num_read_kb":41433,"num_write":446,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.14","version":"112'574","reported_seq":5239,"reported_epoch":113,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-18T01:04:34.370216+0000","last_change":"2024-09-18T01:04:34.370169+0000","last_active":"2024-09-18T01:04:34.370216+0000","last_peered":"2024-09-18T01:04:34.370216+0000","last_clean":"2024-09-18T01:04:34.370216+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:04:34.370216+0000","last_undegraded":"2024-09-18T01:04:34.370216+0000","last_fullsized":"2024-09-18T01:04:34.370216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:15:23.560794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168008,"stat_sum":{"num_bytes":3201040,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":11,"num_read":6144,"num_read_kb":17373,"num_write":607,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":209,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.369957+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:04:34.369957+0000","last_peered":"2024-09-18T01:04:34.369957+0000","last_clean":"2024-09-18T01:04:34.369957+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:04:34.369957+0000","last_undegraded":"2024-09-18T01:04:34.369957+0000","last_fullsized":"2024-09-18T01:04:34.369957+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"112'759","reported_seq":7045,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366310+0000","last_change":"2024-09-18T01:04:34.366715+0000","last_active":"2024-09-18T01:04:34.366310+0000","last_peered":"2024-09-18T01:04:34.366310+0000","last_clean":"2024-09-18T01:04:34.366310+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:04:34.366310+0000","last_undegraded":"2024-09-18T01:04:34.366310+0000","last_fullsized":"2024-09-18T01:04:34.366310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:18:43.186443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584345,"stat_sum":{"num_bytes":18325504,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":970,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":207,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.287969+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:04:33.287969+0000","last_peered":"2024-09-18T01:04:33.287969+0000","last_clean":"2024-09-18T01:04:33.287969+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:04:33.287969+0000","last_undegraded":"2024-09-18T01:04:33.287969+0000","last_fullsized":"2024-09-18T01:04:33.287969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'2093","reported_seq":10800,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.289318+0000","last_change":"2024-09-18T01:04:33.289318+0000","last_active":"2024-09-18T01:04:33.289318+0000","last_peered":"2024-09-18T01:04:33.289318+0000","last_clean":"2024-09-18T01:04:33.289318+0000","last_became_active":"2024-09-18T01:03:45.291927+0000","last_became_peered":"2024-09-18T01:03:45.291927+0000","last_unstale":"2024-09-18T01:04:33.289318+0000","last_undegraded":"2024-09-18T01:04:33.289318+0000","last_fullsized":"2024-09-18T01:04:33.289318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2093,"log_dups_size":0,"ondisk_log_size":2093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:33.396897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001082707,"stat_sum":{"num_bytes":1839330,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":19783,"num_read_kb":47240,"num_write":3342,"num_write_kb":9354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":2222926,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.369693+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:04:34.369693+0000","last_peered":"2024-09-18T01:04:34.369693+0000","last_clean":"2024-09-18T01:04:34.369693+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:04:34.369693+0000","last_undegraded":"2024-09-18T01:04:34.369693+0000","last_fullsized":"2024-09-18T01:04:34.369693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"108'457","reported_seq":5974,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288619+0000","last_change":"2024-09-18T01:04:33.288875+0000","last_active":"2024-09-18T01:04:33.288619+0000","last_peered":"2024-09-18T01:04:33.288619+0000","last_clean":"2024-09-18T01:04:33.288619+0000","last_became_active":"2024-09-18T01:03:45.291124+0000","last_became_peered":"2024-09-18T01:03:45.291124+0000","last_unstale":"2024-09-18T01:04:33.288619+0000","last_undegraded":"2024-09-18T01:04:33.288619+0000","last_fullsized":"2024-09-18T01:04:33.288619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:31:30.564600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013366389999999999,"stat_sum":{"num_bytes":9590784,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":5793,"num_read_kb":21958,"num_write":477,"num_write_kb":7398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":270420,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.287729+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:04:33.287729+0000","last_peered":"2024-09-18T01:04:33.287729+0000","last_clean":"2024-09-18T01:04:33.287729+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:04:33.287729+0000","last_undegraded":"2024-09-18T01:04:33.287729+0000","last_fullsized":"2024-09-18T01:04:33.287729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"110'355","reported_seq":8063,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365811+0000","last_change":"2024-09-18T01:04:34.366494+0000","last_active":"2024-09-18T01:04:34.365811+0000","last_peered":"2024-09-18T01:04:34.365811+0000","last_clean":"2024-09-18T01:04:34.365811+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:04:34.365811+0000","last_undegraded":"2024-09-18T01:04:34.365811+0000","last_fullsized":"2024-09-18T01:04:34.365811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:37.898220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096930599999999997,"stat_sum":{"num_bytes":545312,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":7490,"num_read_kb":41288,"num_write":357,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.3","version":"112'1023","reported_seq":7277,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366400+0000","last_change":"2024-09-18T01:04:34.366601+0000","last_active":"2024-09-18T01:04:34.366400+0000","last_peered":"2024-09-18T01:04:34.366400+0000","last_clean":"2024-09-18T01:04:34.366400+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:04:34.366400+0000","last_undegraded":"2024-09-18T01:04:34.366400+0000","last_fullsized":"2024-09-18T01:04:34.366400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1023,"log_dups_size":0,"ondisk_log_size":1023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:41.658713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064156499999999997,"stat_sum":{"num_bytes":444416,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":9667,"num_read_kb":61468,"num_write":1297,"num_write_kb":7914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"2.2","version":"18'2","reported_seq":199,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.287904+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:04:33.287904+0000","last_peered":"2024-09-18T01:04:33.287904+0000","last_clean":"2024-09-18T01:04:33.287904+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:04:33.287904+0000","last_undegraded":"2024-09-18T01:04:33.287904+0000","last_fullsized":"2024-09-18T01:04:33.287904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"112'295","reported_seq":6625,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.387262+0000","last_change":"2024-09-18T01:04:33.288362+0000","last_active":"2024-09-18T01:04:33.387262+0000","last_peered":"2024-09-18T01:04:33.387262+0000","last_clean":"2024-09-18T01:04:33.387262+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:04:33.387262+0000","last_undegraded":"2024-09-18T01:04:33.387262+0000","last_fullsized":"2024-09-18T01:04:33.387262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:52.385272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086043999999999997,"stat_sum":{"num_bytes":25062130,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":6142,"num_read_kb":49427,"num_write":288,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":209,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365368+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:04:34.365368+0000","last_peered":"2024-09-18T01:04:34.365368+0000","last_clean":"2024-09-18T01:04:34.365368+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:04:34.365368+0000","last_undegraded":"2024-09-18T01:04:34.365368+0000","last_fullsized":"2024-09-18T01:04:34.365368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"111'372","reported_seq":4490,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366373+0000","last_change":"2024-09-18T01:04:34.366857+0000","last_active":"2024-09-18T01:04:34.366373+0000","last_peered":"2024-09-18T01:04:34.366373+0000","last_clean":"2024-09-18T01:04:34.366373+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:04:34.366373+0000","last_undegraded":"2024-09-18T01:04:34.366373+0000","last_fullsized":"2024-09-18T01:04:34.366373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:19:11.794983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063822900000000003,"stat_sum":{"num_bytes":1765376,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":314,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":210,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.368716+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:04:34.368716+0000","last_peered":"2024-09-18T01:04:34.368716+0000","last_clean":"2024-09-18T01:04:34.368716+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:04:34.368716+0000","last_undegraded":"2024-09-18T01:04:34.368716+0000","last_fullsized":"2024-09-18T01:04:34.368716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"111'459","reported_seq":6817,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288791+0000","last_change":"2024-09-18T01:04:33.289040+0000","last_active":"2024-09-18T01:04:33.288791+0000","last_peered":"2024-09-18T01:04:33.288791+0000","last_clean":"2024-09-18T01:04:33.288791+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:04:33.288791+0000","last_undegraded":"2024-09-18T01:04:33.288791+0000","last_fullsized":"2024-09-18T01:04:33.288791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:42:41.875066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013381669999999999,"stat_sum":{"num_bytes":1540169,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":7261,"num_read_kb":41803,"num_write":417,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.3","version":"16'1","reported_seq":187,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365608+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:04:34.365608+0000","last_peered":"2024-09-18T01:04:34.365608+0000","last_clean":"2024-09-18T01:04:34.365608+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:04:34.365608+0000","last_undegraded":"2024-09-18T01:04:34.365608+0000","last_fullsized":"2024-09-18T01:04:34.365608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":362,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.368811+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:04:34.368811+0000","last_peered":"2024-09-18T01:04:34.368811+0000","last_clean":"2024-09-18T01:04:34.368811+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:04:34.368811+0000","last_undegraded":"2024-09-18T01:04:34.368811+0000","last_fullsized":"2024-09-18T01:04:34.368811+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"109'652","reported_seq":8254,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366553+0000","last_change":"2024-09-18T01:04:34.366974+0000","last_active":"2024-09-18T01:04:34.366553+0000","last_peered":"2024-09-18T01:04:34.366553+0000","last_clean":"2024-09-18T01:04:34.366553+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:04:34.366553+0000","last_undegraded":"2024-09-18T01:04:34.366553+0000","last_fullsized":"2024-09-18T01:04:34.366553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:33.707516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001260096,"stat_sum":{"num_bytes":12012121,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":8662,"num_read_kb":60970,"num_write":839,"num_write_kb":15561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.9","version":"110'439","reported_seq":4999,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288029+0000","last_change":"2024-09-18T01:04:33.288562+0000","last_active":"2024-09-18T01:04:33.288029+0000","last_peered":"2024-09-18T01:04:33.288029+0000","last_clean":"2024-09-18T01:04:33.288029+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:04:33.288029+0000","last_undegraded":"2024-09-18T01:04:33.288029+0000","last_fullsized":"2024-09-18T01:04:33.288029+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.232658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185064,"stat_sum":{"num_bytes":1538048,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":4910,"num_read_kb":30805,"num_write":517,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.a","version":"113'729","reported_seq":5266,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370068+0000","last_change":"2024-09-18T01:04:34.370294+0000","last_active":"2024-09-18T01:04:34.370068+0000","last_peered":"2024-09-18T01:04:34.370068+0000","last_clean":"2024-09-18T01:04:34.370068+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:04:34.370068+0000","last_undegraded":"2024-09-18T01:04:34.370068+0000","last_fullsized":"2024-09-18T01:04:34.370068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:17:16.597258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013687459999999999,"stat_sum":{"num_bytes":3059747,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":6591,"num_read_kb":22736,"num_write":904,"num_write_kb":4908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.b","version":"111'508","reported_seq":6860,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.410572+0000","last_change":"2024-09-18T01:04:33.288702+0000","last_active":"2024-09-18T01:04:33.410572+0000","last_peered":"2024-09-18T01:04:33.410572+0000","last_clean":"2024-09-18T01:04:33.410572+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:04:33.410572+0000","last_undegraded":"2024-09-18T01:04:33.410572+0000","last_fullsized":"2024-09-18T01:04:33.410572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:46.341832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001188387,"stat_sum":{"num_bytes":389136,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":420,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.c","version":"111'368","reported_seq":5972,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366925+0000","last_change":"2024-09-18T01:04:34.367232+0000","last_active":"2024-09-18T01:04:34.366925+0000","last_peered":"2024-09-18T01:04:34.366925+0000","last_clean":"2024-09-18T01:04:34.366925+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:04:34.366925+0000","last_undegraded":"2024-09-18T01:04:34.366925+0000","last_fullsized":"2024-09-18T01:04:34.366925+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:07.020202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086295299999999996,"stat_sum":{"num_bytes":361472,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":313,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.d","version":"112'523","reported_seq":6104,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370333+0000","last_change":"2024-09-18T01:04:34.370631+0000","last_active":"2024-09-18T01:04:34.370333+0000","last_peered":"2024-09-18T01:04:34.370333+0000","last_clean":"2024-09-18T01:04:34.370333+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:04:34.370333+0000","last_undegraded":"2024-09-18T01:04:34.370333+0000","last_fullsized":"2024-09-18T01:04:34.370333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:33.246390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010653850000000001,"stat_sum":{"num_bytes":4695552,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":6693,"num_read_kb":28822,"num_write":530,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.e","version":"111'393","reported_seq":6788,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366647+0000","last_change":"2024-09-18T01:04:34.366955+0000","last_active":"2024-09-18T01:04:34.366647+0000","last_peered":"2024-09-18T01:04:34.366647+0000","last_clean":"2024-09-18T01:04:34.366647+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:04:34.366647+0000","last_undegraded":"2024-09-18T01:04:34.366647+0000","last_fullsized":"2024-09-18T01:04:34.366647+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:00:03.815625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098131999999999994,"stat_sum":{"num_bytes":13923840,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":336,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.f","version":"112'583","reported_seq":13069,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370376+0000","last_change":"2024-09-18T01:04:34.370706+0000","last_active":"2024-09-18T01:04:34.370376+0000","last_peered":"2024-09-18T01:04:34.370376+0000","last_clean":"2024-09-18T01:04:34.370376+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:04:34.370376+0000","last_undegraded":"2024-09-18T01:04:34.370376+0000","last_fullsized":"2024-09-18T01:04:34.370376+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:37.161315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010786319999999999,"stat_sum":{"num_bytes":3105280,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":10937,"num_read_kb":25876,"num_write":428,"num_write_kb":7503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.10","version":"112'433","reported_seq":4514,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.445648+0000","last_change":"2024-09-18T01:04:33.287990+0000","last_active":"2024-09-18T01:04:33.445648+0000","last_peered":"2024-09-18T01:04:33.445648+0000","last_clean":"2024-09-18T01:04:33.445648+0000","last_became_active":"2024-09-18T01:03:45.291335+0000","last_became_peered":"2024-09-18T01:03:45.291335+0000","last_unstale":"2024-09-18T01:04:33.445648+0000","last_undegraded":"2024-09-18T01:04:33.445648+0000","last_fullsized":"2024-09-18T01:04:33.445648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:31.964008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064601199999999995,"stat_sum":{"num_bytes":2872358,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":5012,"num_read_kb":25885,"num_write":428,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2210583,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.11","version":"106'381","reported_seq":4319,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.369059+0000","last_change":"2024-09-18T01:04:34.369382+0000","last_active":"2024-09-18T01:04:34.369059+0000","last_peered":"2024-09-18T01:04:34.369059+0000","last_clean":"2024-09-18T01:04:34.369059+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:04:34.369059+0000","last_undegraded":"2024-09-18T01:04:34.369059+0000","last_fullsized":"2024-09-18T01:04:34.369059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:10.247607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000489271,"stat_sum":{"num_bytes":428544,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":4079,"num_read_kb":19488,"num_write":366,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.12","version":"111'960","reported_seq":10378,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.367288+0000","last_change":"2024-09-18T01:04:34.367514+0000","last_active":"2024-09-18T01:04:34.367288+0000","last_peered":"2024-09-18T01:04:34.367288+0000","last_clean":"2024-09-18T01:04:34.367288+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:04:34.367288+0000","last_undegraded":"2024-09-18T01:04:34.367288+0000","last_fullsized":"2024-09-18T01:04:34.367288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:57.603173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010986780000000001,"stat_sum":{"num_bytes":22406235,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":12763,"num_read_kb":52311,"num_write":1218,"num_write_kb":15851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.13","version":"110'305","reported_seq":3831,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370118+0000","last_change":"2024-09-18T01:04:34.370286+0000","last_active":"2024-09-18T01:04:34.370118+0000","last_peered":"2024-09-18T01:04:34.370118+0000","last_clean":"2024-09-18T01:04:34.370118+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:04:34.370118+0000","last_undegraded":"2024-09-18T01:04:34.370118+0000","last_fullsized":"2024-09-18T01:04:34.370118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:19.111841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012078970000000001,"stat_sum":{"num_bytes":10030080,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":3257,"num_read_kb":13085,"num_write":291,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":208512971,"num_objects":385,"num_object_clones":23,"num_object_copies":770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":385,"num_whiteouts":194,"num_read":263767,"num_read_kb":1198902,"num_write":22492,"num_write_kb":286306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":793,"num_bytes_recovered":31931717,"num_keys_recovered":73,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":6237078,"data_compressed":142280,"data_compressed_allocated":638976,"data_compressed_original":4117504,"omap_allocated":0,"internal_metadata":0},"log_size":20089,"ondisk_log_size":20089,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640284,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640286,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63704,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308136,"statfs":{"total":96636764160,"available":96571531264,"internally_reserved":0,"allocated":2027520,"data_stored":1678873,"data_compressed":3634,"data_compressed_allocated":90112,"data_compressed_original":729088,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47456,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324384,"statfs":{"total":96636764160,"available":96588169216,"internally_reserved":0,"allocated":2433024,"data_stored":2772899,"data_compressed":74618,"data_compressed_allocated":225280,"data_compressed_original":1635328,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640284,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73188,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298652,"statfs":{"total":96636764160,"available":96561819648,"internally_reserved":0,"allocated":1781760,"data_stored":1297136,"data_compressed":4734,"data_compressed_allocated":69632,"data_compressed_original":522752,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65396,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306444,"statfs":{"total":96636764160,"available":96569798656,"internally_reserved":0,"allocated":2318336,"data_stored":2019743,"data_compressed":4768,"data_compressed_allocated":131072,"data_compressed_original":917504,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640285,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59608,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312232,"statfs":{"total":96636764160,"available":96575725568,"internally_reserved":0,"allocated":2097152,"data_stored":2374796,"data_compressed":71626,"data_compressed_allocated":196608,"data_compressed_original":1459712,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":708608,"data_stored":1239290,"data_compressed":63076,"data_compressed_allocated":159744,"data_compressed_original":886272,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":987136,"data_stored":1474605,"data_compressed":4768,"data_compressed_allocated":131072,"data_compressed_original":917504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":752017,"data_compressed":4734,"data_compressed_allocated":69632,"data_compressed_original":522752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1637412,"data_compressed":66068,"data_compressed_allocated":188416,"data_compressed_original":1061888,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":700416,"data_stored":1133754,"data_compressed":3634,"data_compressed_allocated":90112,"data_compressed_original":729088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:04:39.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:04:39.710 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:39.709+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:39.710 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.710 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.719 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:39.811 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:39.809+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:39.863 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.863 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.872 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:39.911 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:39.910+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:39.975 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:39.976 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:39.984 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:40.028 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:40.027+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:40.039 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:40.040 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:04:40.053 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-18T01:04:39.295882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209103358,"num_objects":389,"num_object_clones":23,"num_object_copies":778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":194,"num_read":263884,"num_read_kb":1199003,"num_write":22653,"num_write_kb":288724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":795,"num_bytes_recovered":31931736,"num_keys_recovered":73,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20209,"ondisk_log_size":20209,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":309352,"kb_used_data":10408,"kb_used_omap":7,"kb_used_meta":298808,"kb_avail":471549848,"statfs":{"total":483183820800,"available":482867044352,"internally_reserved":0,"allocated":10657792,"data_stored":10143447,"data_compressed":159380,"data_compressed_allocated":712704,"data_compressed_original":5264384,"omap_allocated":7973,"internal_metadata":305979611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.930137"},"pg_stats":[{"pgid":"3.1f","version":"110'647","reported_seq":10459,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370731+0000","last_change":"2024-09-18T01:04:34.370950+0000","last_active":"2024-09-18T01:04:34.370731+0000","last_peered":"2024-09-18T01:04:34.370731+0000","last_clean":"2024-09-18T01:04:34.370731+0000","last_became_active":"2024-09-18T01:04:21.303421+0000","last_became_peered":"2024-09-18T01:04:21.303421+0000","last_unstale":"2024-09-18T01:04:34.370731+0000","last_undegraded":"2024-09-18T01:04:34.370731+0000","last_fullsized":"2024-09-18T01:04:34.370731+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:29:31.129611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011175670000000001,"stat_sum":{"num_bytes":9754128,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":14,"num_read":12632,"num_read_kb":58132,"num_write":536,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1e","version":"111'1472","reported_seq":7646,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365748+0000","last_change":"2024-09-18T01:04:34.366154+0000","last_active":"2024-09-18T01:04:34.365748+0000","last_peered":"2024-09-18T01:04:34.365748+0000","last_clean":"2024-09-18T01:04:34.365748+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:04:34.365748+0000","last_undegraded":"2024-09-18T01:04:34.365748+0000","last_fullsized":"2024-09-18T01:04:34.365748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1472,"log_dups_size":0,"ondisk_log_size":1472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:48.922070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064141899999999995,"stat_sum":{"num_bytes":89088,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2011,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1d","version":"112'436","reported_seq":7343,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366703+0000","last_change":"2024-09-18T01:04:34.367041+0000","last_active":"2024-09-18T01:04:34.366703+0000","last_peered":"2024-09-18T01:04:34.366703+0000","last_clean":"2024-09-18T01:04:34.366703+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:04:34.366703+0000","last_undegraded":"2024-09-18T01:04:34.366703+0000","last_fullsized":"2024-09-18T01:04:34.366703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:55.314450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094235900000000004,"stat_sum":{"num_bytes":3716283,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":7596,"num_read_kb":29158,"num_write":416,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1c","version":"112'834","reported_seq":12561,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366174+0000","last_change":"2024-09-18T01:04:34.366358+0000","last_active":"2024-09-18T01:04:34.366174+0000","last_peered":"2024-09-18T01:04:34.366174+0000","last_clean":"2024-09-18T01:04:34.366174+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:04:34.366174+0000","last_undegraded":"2024-09-18T01:04:34.366174+0000","last_fullsized":"2024-09-18T01:04:34.366174+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:26:26.424573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041547200000000001,"stat_sum":{"num_bytes":11531817,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":15622,"num_read_kb":43359,"num_write":701,"num_write_kb":8643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1b","version":"108'621","reported_seq":4435,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366461+0000","last_change":"2024-09-18T01:04:34.366650+0000","last_active":"2024-09-18T01:04:34.366461+0000","last_peered":"2024-09-18T01:04:34.366461+0000","last_clean":"2024-09-18T01:04:34.366461+0000","last_became_active":"2024-09-18T01:03:44.290192+0000","last_became_peered":"2024-09-18T01:03:44.290192+0000","last_unstale":"2024-09-18T01:04:34.366461+0000","last_undegraded":"2024-09-18T01:04:34.366461+0000","last_fullsized":"2024-09-18T01:04:34.366461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:24.341556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073620999999999995,"stat_sum":{"num_bytes":8620129,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":5106,"num_read_kb":19572,"num_write":767,"num_write_kb":5929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":1672836,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.1a","version":"109'232","reported_seq":5907,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366121+0000","last_change":"2024-09-18T01:04:34.366344+0000","last_active":"2024-09-18T01:04:34.366121+0000","last_peered":"2024-09-18T01:04:34.366121+0000","last_clean":"2024-09-18T01:04:34.366121+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:04:34.366121+0000","last_undegraded":"2024-09-18T01:04:34.366121+0000","last_fullsized":"2024-09-18T01:04:34.366121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:28.557561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042838899999999999,"stat_sum":{"num_bytes":3674643,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":5403,"num_read_kb":19381,"num_write":228,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.19","version":"111'622","reported_seq":8531,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288960+0000","last_change":"2024-09-18T01:04:33.289281+0000","last_active":"2024-09-18T01:04:33.288960+0000","last_peered":"2024-09-18T01:04:33.288960+0000","last_clean":"2024-09-18T01:04:33.288960+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:04:33.288960+0000","last_undegraded":"2024-09-18T01:04:33.288960+0000","last_fullsized":"2024-09-18T01:04:33.288960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:22.983717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116394,"stat_sum":{"num_bytes":238633,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":10840,"num_read_kb":39965,"num_write":576,"num_write_kb":9910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.18","version":"112'734","reported_seq":8525,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366247+0000","last_change":"2024-09-18T01:04:34.367003+0000","last_active":"2024-09-18T01:04:34.366247+0000","last_peered":"2024-09-18T01:04:34.366247+0000","last_clean":"2024-09-18T01:04:34.366247+0000","last_became_active":"2024-09-18T01:03:45.291065+0000","last_became_peered":"2024-09-18T01:03:45.291065+0000","last_unstale":"2024-09-18T01:04:34.366247+0000","last_undegraded":"2024-09-18T01:04:34.366247+0000","last_fullsized":"2024-09-18T01:04:34.366247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:47.303745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013472930000000001,"stat_sum":{"num_bytes":3448832,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":9266,"num_read_kb":83025,"num_write":701,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":11170357,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.17","version":"111'893","reported_seq":8180,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366865+0000","last_change":"2024-09-18T01:04:34.367318+0000","last_active":"2024-09-18T01:04:34.366865+0000","last_peered":"2024-09-18T01:04:34.366865+0000","last_clean":"2024-09-18T01:04:34.366865+0000","last_became_active":"2024-09-18T01:03:45.291814+0000","last_became_peered":"2024-09-18T01:03:45.291814+0000","last_unstale":"2024-09-18T01:04:34.366865+0000","last_undegraded":"2024-09-18T01:04:34.366865+0000","last_fullsized":"2024-09-18T01:04:34.366865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:42.308373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099354200000000004,"stat_sum":{"num_bytes":16832548,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":10622,"num_read_kb":32379,"num_write":1044,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1702477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.16","version":"112'357","reported_seq":3775,"reported_epoch":113,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-18T01:04:34.369979+0000","last_change":"2024-09-18T01:04:34.369963+0000","last_active":"2024-09-18T01:04:34.369979+0000","last_peered":"2024-09-18T01:04:34.369979+0000","last_clean":"2024-09-18T01:04:34.369979+0000","last_became_active":"2024-09-18T01:03:38.521858+0000","last_became_peered":"2024-09-18T01:03:38.521858+0000","last_unstale":"2024-09-18T01:04:34.369979+0000","last_undegraded":"2024-09-18T01:04:34.369979+0000","last_fullsized":"2024-09-18T01:04:34.369979+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:43.719181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055879699999999998,"stat_sum":{"num_bytes":4040289,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":3213,"num_read_kb":32432,"num_write":407,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.15","version":"111'580","reported_seq":9797,"reported_epoch":113,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-18T01:04:34.369785+0000","last_change":"2024-09-18T01:04:34.369768+0000","last_active":"2024-09-18T01:04:34.369785+0000","last_peered":"2024-09-18T01:04:34.369785+0000","last_clean":"2024-09-18T01:04:34.369785+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:04:34.369785+0000","last_undegraded":"2024-09-18T01:04:34.369785+0000","last_fullsized":"2024-09-18T01:04:34.369785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:39:32.525607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015372529999999999,"stat_sum":{"num_bytes":9430107,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":11665,"num_read_kb":41433,"num_write":446,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.14","version":"112'574","reported_seq":5239,"reported_epoch":113,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-18T01:04:34.370216+0000","last_change":"2024-09-18T01:04:34.370169+0000","last_active":"2024-09-18T01:04:34.370216+0000","last_peered":"2024-09-18T01:04:34.370216+0000","last_clean":"2024-09-18T01:04:34.370216+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:04:34.370216+0000","last_undegraded":"2024-09-18T01:04:34.370216+0000","last_fullsized":"2024-09-18T01:04:34.370216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:15:23.560794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00168008,"stat_sum":{"num_bytes":3201040,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":11,"num_read":6144,"num_read_kb":17373,"num_write":607,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":209,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.369957+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:04:34.369957+0000","last_peered":"2024-09-18T01:04:34.369957+0000","last_clean":"2024-09-18T01:04:34.369957+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:04:34.369957+0000","last_undegraded":"2024-09-18T01:04:34.369957+0000","last_fullsized":"2024-09-18T01:04:34.369957+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"112'759","reported_seq":7045,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366310+0000","last_change":"2024-09-18T01:04:34.366715+0000","last_active":"2024-09-18T01:04:34.366310+0000","last_peered":"2024-09-18T01:04:34.366310+0000","last_clean":"2024-09-18T01:04:34.366310+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:04:34.366310+0000","last_undegraded":"2024-09-18T01:04:34.366310+0000","last_fullsized":"2024-09-18T01:04:34.366310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:18:43.186443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000584345,"stat_sum":{"num_bytes":18325504,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":970,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":207,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.287969+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:04:33.287969+0000","last_peered":"2024-09-18T01:04:33.287969+0000","last_clean":"2024-09-18T01:04:33.287969+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:04:33.287969+0000","last_undegraded":"2024-09-18T01:04:33.287969+0000","last_fullsized":"2024-09-18T01:04:33.287969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"112'2093","reported_seq":10800,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.289318+0000","last_change":"2024-09-18T01:04:33.289318+0000","last_active":"2024-09-18T01:04:33.289318+0000","last_peered":"2024-09-18T01:04:33.289318+0000","last_clean":"2024-09-18T01:04:33.289318+0000","last_became_active":"2024-09-18T01:03:45.291927+0000","last_became_peered":"2024-09-18T01:03:45.291927+0000","last_unstale":"2024-09-18T01:04:33.289318+0000","last_undegraded":"2024-09-18T01:04:33.289318+0000","last_fullsized":"2024-09-18T01:04:33.289318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2093,"log_dups_size":0,"ondisk_log_size":2093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:33.396897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001082707,"stat_sum":{"num_bytes":1839330,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":19783,"num_read_kb":47240,"num_write":3342,"num_write_kb":9354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":2222926,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.369693+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:04:34.369693+0000","last_peered":"2024-09-18T01:04:34.369693+0000","last_clean":"2024-09-18T01:04:34.369693+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:04:34.369693+0000","last_undegraded":"2024-09-18T01:04:34.369693+0000","last_fullsized":"2024-09-18T01:04:34.369693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"108'457","reported_seq":5974,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288619+0000","last_change":"2024-09-18T01:04:33.288875+0000","last_active":"2024-09-18T01:04:33.288619+0000","last_peered":"2024-09-18T01:04:33.288619+0000","last_clean":"2024-09-18T01:04:33.288619+0000","last_became_active":"2024-09-18T01:03:45.291124+0000","last_became_peered":"2024-09-18T01:03:45.291124+0000","last_unstale":"2024-09-18T01:04:33.288619+0000","last_undegraded":"2024-09-18T01:04:33.288619+0000","last_fullsized":"2024-09-18T01:04:33.288619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:31:30.564600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013366389999999999,"stat_sum":{"num_bytes":9590784,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":5793,"num_read_kb":21958,"num_write":477,"num_write_kb":7398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":270420,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":207,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.287729+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:04:33.287729+0000","last_peered":"2024-09-18T01:04:33.287729+0000","last_clean":"2024-09-18T01:04:33.287729+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:04:33.287729+0000","last_undegraded":"2024-09-18T01:04:33.287729+0000","last_fullsized":"2024-09-18T01:04:33.287729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"110'355","reported_seq":8063,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365811+0000","last_change":"2024-09-18T01:04:34.366494+0000","last_active":"2024-09-18T01:04:34.365811+0000","last_peered":"2024-09-18T01:04:34.365811+0000","last_clean":"2024-09-18T01:04:34.365811+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:04:34.365811+0000","last_undegraded":"2024-09-18T01:04:34.365811+0000","last_fullsized":"2024-09-18T01:04:34.365811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:37.898220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096930599999999997,"stat_sum":{"num_bytes":545312,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":7490,"num_read_kb":41288,"num_write":357,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.3","version":"112'1023","reported_seq":7277,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366400+0000","last_change":"2024-09-18T01:04:34.366601+0000","last_active":"2024-09-18T01:04:34.366400+0000","last_peered":"2024-09-18T01:04:34.366400+0000","last_clean":"2024-09-18T01:04:34.366400+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:04:34.366400+0000","last_undegraded":"2024-09-18T01:04:34.366400+0000","last_fullsized":"2024-09-18T01:04:34.366400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1023,"log_dups_size":0,"ondisk_log_size":1023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:30:41.658713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064156499999999997,"stat_sum":{"num_bytes":444416,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":9667,"num_read_kb":61468,"num_write":1297,"num_write_kb":7914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"2.2","version":"18'2","reported_seq":199,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.287904+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:04:33.287904+0000","last_peered":"2024-09-18T01:04:33.287904+0000","last_clean":"2024-09-18T01:04:33.287904+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:04:33.287904+0000","last_undegraded":"2024-09-18T01:04:33.287904+0000","last_fullsized":"2024-09-18T01:04:33.287904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"112'295","reported_seq":6625,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.387262+0000","last_change":"2024-09-18T01:04:33.288362+0000","last_active":"2024-09-18T01:04:33.387262+0000","last_peered":"2024-09-18T01:04:33.387262+0000","last_clean":"2024-09-18T01:04:33.387262+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:04:33.387262+0000","last_undegraded":"2024-09-18T01:04:33.387262+0000","last_fullsized":"2024-09-18T01:04:33.387262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:52.385272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086043999999999997,"stat_sum":{"num_bytes":25062130,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":6142,"num_read_kb":49427,"num_write":288,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":209,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365368+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:04:34.365368+0000","last_peered":"2024-09-18T01:04:34.365368+0000","last_clean":"2024-09-18T01:04:34.365368+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:04:34.365368+0000","last_undegraded":"2024-09-18T01:04:34.365368+0000","last_fullsized":"2024-09-18T01:04:34.365368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"111'372","reported_seq":4490,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366373+0000","last_change":"2024-09-18T01:04:34.366857+0000","last_active":"2024-09-18T01:04:34.366373+0000","last_peered":"2024-09-18T01:04:34.366373+0000","last_clean":"2024-09-18T01:04:34.366373+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:04:34.366373+0000","last_undegraded":"2024-09-18T01:04:34.366373+0000","last_fullsized":"2024-09-18T01:04:34.366373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:19:11.794983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063822900000000003,"stat_sum":{"num_bytes":1765376,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":314,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":210,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.368716+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:04:34.368716+0000","last_peered":"2024-09-18T01:04:34.368716+0000","last_clean":"2024-09-18T01:04:34.368716+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:04:34.368716+0000","last_undegraded":"2024-09-18T01:04:34.368716+0000","last_fullsized":"2024-09-18T01:04:34.368716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"111'459","reported_seq":6817,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288791+0000","last_change":"2024-09-18T01:04:33.289040+0000","last_active":"2024-09-18T01:04:33.288791+0000","last_peered":"2024-09-18T01:04:33.288791+0000","last_clean":"2024-09-18T01:04:33.288791+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:04:33.288791+0000","last_undegraded":"2024-09-18T01:04:33.288791+0000","last_fullsized":"2024-09-18T01:04:33.288791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:42:41.875066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013381669999999999,"stat_sum":{"num_bytes":1540169,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":7261,"num_read_kb":41803,"num_write":417,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"2.3","version":"16'1","reported_seq":187,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.365608+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:04:34.365608+0000","last_peered":"2024-09-18T01:04:34.365608+0000","last_clean":"2024-09-18T01:04:34.365608+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:04:34.365608+0000","last_undegraded":"2024-09-18T01:04:34.365608+0000","last_fullsized":"2024-09-18T01:04:34.365608+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":362,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.368811+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:04:34.368811+0000","last_peered":"2024-09-18T01:04:34.368811+0000","last_clean":"2024-09-18T01:04:34.368811+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:04:34.368811+0000","last_undegraded":"2024-09-18T01:04:34.368811+0000","last_fullsized":"2024-09-18T01:04:34.368811+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"109'652","reported_seq":8254,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366553+0000","last_change":"2024-09-18T01:04:34.366974+0000","last_active":"2024-09-18T01:04:34.366553+0000","last_peered":"2024-09-18T01:04:34.366553+0000","last_clean":"2024-09-18T01:04:34.366553+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:04:34.366553+0000","last_undegraded":"2024-09-18T01:04:34.366553+0000","last_fullsized":"2024-09-18T01:04:34.366553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:36:33.707516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001260096,"stat_sum":{"num_bytes":12012121,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":8662,"num_read_kb":60970,"num_write":839,"num_write_kb":15561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.9","version":"110'439","reported_seq":4999,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.288029+0000","last_change":"2024-09-18T01:04:33.288562+0000","last_active":"2024-09-18T01:04:33.288029+0000","last_peered":"2024-09-18T01:04:33.288029+0000","last_clean":"2024-09-18T01:04:33.288029+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:04:33.288029+0000","last_undegraded":"2024-09-18T01:04:33.288029+0000","last_fullsized":"2024-09-18T01:04:33.288029+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:01.232658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185064,"stat_sum":{"num_bytes":1538048,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":4910,"num_read_kb":30805,"num_write":517,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.a","version":"113'729","reported_seq":5266,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370068+0000","last_change":"2024-09-18T01:04:34.370294+0000","last_active":"2024-09-18T01:04:34.370068+0000","last_peered":"2024-09-18T01:04:34.370068+0000","last_clean":"2024-09-18T01:04:34.370068+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:04:34.370068+0000","last_undegraded":"2024-09-18T01:04:34.370068+0000","last_fullsized":"2024-09-18T01:04:34.370068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:17:16.597258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013687459999999999,"stat_sum":{"num_bytes":3059747,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":6591,"num_read_kb":22736,"num_write":904,"num_write_kb":4908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.b","version":"111'508","reported_seq":6860,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.410572+0000","last_change":"2024-09-18T01:04:33.288702+0000","last_active":"2024-09-18T01:04:33.410572+0000","last_peered":"2024-09-18T01:04:33.410572+0000","last_clean":"2024-09-18T01:04:33.410572+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:04:33.410572+0000","last_undegraded":"2024-09-18T01:04:33.410572+0000","last_fullsized":"2024-09-18T01:04:33.410572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:46.341832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001188387,"stat_sum":{"num_bytes":389136,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":420,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.c","version":"111'368","reported_seq":5972,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366925+0000","last_change":"2024-09-18T01:04:34.367232+0000","last_active":"2024-09-18T01:04:34.366925+0000","last_peered":"2024-09-18T01:04:34.366925+0000","last_clean":"2024-09-18T01:04:34.366925+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:04:34.366925+0000","last_undegraded":"2024-09-18T01:04:34.366925+0000","last_fullsized":"2024-09-18T01:04:34.366925+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:07.020202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086295299999999996,"stat_sum":{"num_bytes":361472,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":313,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.d","version":"112'523","reported_seq":6104,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370333+0000","last_change":"2024-09-18T01:04:34.370631+0000","last_active":"2024-09-18T01:04:34.370333+0000","last_peered":"2024-09-18T01:04:34.370333+0000","last_clean":"2024-09-18T01:04:34.370333+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:04:34.370333+0000","last_undegraded":"2024-09-18T01:04:34.370333+0000","last_fullsized":"2024-09-18T01:04:34.370333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:33.246390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010653850000000001,"stat_sum":{"num_bytes":4695552,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":6693,"num_read_kb":28822,"num_write":530,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.e","version":"111'393","reported_seq":6788,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.366647+0000","last_change":"2024-09-18T01:04:34.366955+0000","last_active":"2024-09-18T01:04:34.366647+0000","last_peered":"2024-09-18T01:04:34.366647+0000","last_clean":"2024-09-18T01:04:34.366647+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:04:34.366647+0000","last_undegraded":"2024-09-18T01:04:34.366647+0000","last_fullsized":"2024-09-18T01:04:34.366647+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:00:03.815625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098131999999999994,"stat_sum":{"num_bytes":13923840,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":336,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.f","version":"112'583","reported_seq":13069,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370376+0000","last_change":"2024-09-18T01:04:34.370706+0000","last_active":"2024-09-18T01:04:34.370376+0000","last_peered":"2024-09-18T01:04:34.370376+0000","last_clean":"2024-09-18T01:04:34.370376+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:04:34.370376+0000","last_undegraded":"2024-09-18T01:04:34.370376+0000","last_fullsized":"2024-09-18T01:04:34.370376+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:37.161315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010786319999999999,"stat_sum":{"num_bytes":3105280,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":10937,"num_read_kb":25876,"num_write":428,"num_write_kb":7503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.10","version":"112'433","reported_seq":4514,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-18T01:04:33.445648+0000","last_change":"2024-09-18T01:04:33.287990+0000","last_active":"2024-09-18T01:04:33.445648+0000","last_peered":"2024-09-18T01:04:33.445648+0000","last_clean":"2024-09-18T01:04:33.445648+0000","last_became_active":"2024-09-18T01:03:45.291335+0000","last_became_peered":"2024-09-18T01:03:45.291335+0000","last_unstale":"2024-09-18T01:04:33.445648+0000","last_undegraded":"2024-09-18T01:04:33.445648+0000","last_fullsized":"2024-09-18T01:04:33.445648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:31.964008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064601199999999995,"stat_sum":{"num_bytes":2872358,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":5012,"num_read_kb":25885,"num_write":428,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":2210583,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"5"}]},{"pgid":"3.11","version":"106'381","reported_seq":4319,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.369059+0000","last_change":"2024-09-18T01:04:34.369382+0000","last_active":"2024-09-18T01:04:34.369059+0000","last_peered":"2024-09-18T01:04:34.369059+0000","last_clean":"2024-09-18T01:04:34.369059+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:04:34.369059+0000","last_undegraded":"2024-09-18T01:04:34.369059+0000","last_fullsized":"2024-09-18T01:04:34.369059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:10.247607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000489271,"stat_sum":{"num_bytes":428544,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":4079,"num_read_kb":19488,"num_write":366,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.12","version":"111'960","reported_seq":10378,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.367288+0000","last_change":"2024-09-18T01:04:34.367514+0000","last_active":"2024-09-18T01:04:34.367288+0000","last_peered":"2024-09-18T01:04:34.367288+0000","last_clean":"2024-09-18T01:04:34.367288+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:04:34.367288+0000","last_undegraded":"2024-09-18T01:04:34.367288+0000","last_fullsized":"2024-09-18T01:04:34.367288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:57.603173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010986780000000001,"stat_sum":{"num_bytes":22406235,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":12763,"num_read_kb":52311,"num_write":1218,"num_write_kb":15851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"6"}]},{"pgid":"3.13","version":"110'305","reported_seq":3831,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-18T01:04:34.370118+0000","last_change":"2024-09-18T01:04:34.370286+0000","last_active":"2024-09-18T01:04:34.370118+0000","last_peered":"2024-09-18T01:04:34.370118+0000","last_clean":"2024-09-18T01:04:34.370118+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:04:34.370118+0000","last_undegraded":"2024-09-18T01:04:34.370118+0000","last_fullsized":"2024-09-18T01:04:34.370118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:19.111841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012078970000000001,"stat_sum":{"num_bytes":10030080,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":3257,"num_read_kb":13085,"num_write":291,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":208512971,"num_objects":385,"num_object_clones":23,"num_object_copies":770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":385,"num_whiteouts":194,"num_read":263767,"num_read_kb":1198902,"num_write":22492,"num_write_kb":286306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":793,"num_bytes_recovered":31931717,"num_keys_recovered":73,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":6237078,"data_compressed":142280,"data_compressed_allocated":638976,"data_compressed_original":4117504,"omap_allocated":0,"internal_metadata":0},"log_size":20089,"ondisk_log_size":20089,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640284,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640286,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63704,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308136,"statfs":{"total":96636764160,"available":96571531264,"internally_reserved":0,"allocated":2027520,"data_stored":1678873,"data_compressed":3634,"data_compressed_allocated":90112,"data_compressed_original":729088,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47456,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324384,"statfs":{"total":96636764160,"available":96588169216,"internally_reserved":0,"allocated":2433024,"data_stored":2772899,"data_compressed":74618,"data_compressed_allocated":225280,"data_compressed_original":1635328,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640284,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73188,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298652,"statfs":{"total":96636764160,"available":96561819648,"internally_reserved":0,"allocated":1781760,"data_stored":1297136,"data_compressed":4734,"data_compressed_allocated":69632,"data_compressed_original":522752,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65396,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306444,"statfs":{"total":96636764160,"available":96569798656,"internally_reserved":0,"allocated":2318336,"data_stored":2019743,"data_compressed":4768,"data_compressed_allocated":131072,"data_compressed_original":917504,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640285,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59608,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312232,"statfs":{"total":96636764160,"available":96575725568,"internally_reserved":0,"allocated":2097152,"data_stored":2374796,"data_compressed":71626,"data_compressed_allocated":196608,"data_compressed_original":1459712,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":708608,"data_stored":1239290,"data_compressed":63076,"data_compressed_allocated":159744,"data_compressed_original":886272,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":987136,"data_stored":1474605,"data_compressed":4768,"data_compressed_allocated":131072,"data_compressed_original":917504,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":752017,"data_compressed":4734,"data_compressed_allocated":69632,"data_compressed_original":522752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1637412,"data_compressed":66068,"data_compressed_allocated":188416,"data_compressed_original":1061888,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":700416,"data_stored":1133754,"data_compressed":3634,"data_compressed_allocated":90112,"data_compressed_original":729088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:04:40.055 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:04:40.055 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:04:40.055 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T01:04:40.055 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T01:04:40.086 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:40.086 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:40.095 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:40.136 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:40.135+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:40.214 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:40.213+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:40.238 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:40.238 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:40.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:40.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T01:04:40.314 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:40.313+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:40.357 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:40.357 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:40.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 118, 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "62b46b83-8ef2-466f-b28b-2d97f265422e", 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-18T01:02:13.493841+0000", 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-18T01:04:39.286867+0000", 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-18T01:02:24.290620+0000", 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-18T01:02:48.274416+0000", 2024-09-18T01:04:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T01:04:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:24.388064+0000", 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-18T01:04:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-18T01:04:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-18T01:04:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-18T01:04:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-18T01:04:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:30.676199+0000", 2024-09-18T01:04:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-18T01:04:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:04:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2699999809265137, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2699999809265137, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-18T01:02:40.390732+0000", 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-18T01:04:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-18T01:04:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "114", 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T01:04:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-18T01:04:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-18T01:04:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 33, 2024-09-18T01:04:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 114, 2024-09-18T01:04:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-18T01:04:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-18T01:04:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-18T01:04:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T01:04:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-18T01:04:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.5099999904632568, 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.5099999904632568, 2024-09-18T01:04:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.82999998331069946, 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.25, 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.25, 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:04:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4563fb16-8745-4e75-b654-1e96403d91b9", 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 117, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-18T01:04:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1626166637 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/1626166637", 2024-09-18T01:04:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/1626166637", 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/1626166637", 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/1626166637", 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "634965cb-d444-4721-b34a-afbc49105cab", 2024-09-18T01:04:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 117, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2210970259 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/2210970259", 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/2210970259", 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/2210970259", 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/2210970259", 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "83443eab-2fa4-4c7e-a459-edb1bb0ed8d0", 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 117, 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-18T01:04:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 617133675 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/617133675", 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/617133675", 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/617133675", 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/617133675", 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:04:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "5797982a-71c1-4d6a-9604-00fbc6bd16d9", 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 116, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6808", 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6809", 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6810", 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6811", 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6814", 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6815", 2024-09-18T01:04:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6812", 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6813", 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 151031464 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6809/151031464", 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6811/151031464", 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6815/151031464", 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6813/151031464", 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f79e3329-5481-4f33-8d28-6386af186492", 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 74, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6816", 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6817", 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6818", 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6819", 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6822", 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6823", 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6820", 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6821", 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1499036965 2024-09-18T01:04:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6817/1499036965", 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6819/1499036965", 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6823/1499036965", 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6821/1499036965", 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "8c61d2ec-b111-4783-a016-336ed33a0694", 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 20, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6800", 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6801", 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6802", 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6803", 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6806", 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6807", 2024-09-18T01:04:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6804", 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.98:6805", 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2310498524 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.98:6801/2310498524", 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.98:6803/2310498524", 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.98:6807/2310498524", 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.98:6805/2310498524", 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-18T01:04:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.877700+0000", 2024-09-18T01:04:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.360930+0000", 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.403205+0000", 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.952123+0000", 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:22.099545+0000", 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-18T01:04:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-18T01:02:21.694132+0000", 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.9", 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:04:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: 1, 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: 0 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [ 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.4", 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.7", 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: 0 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 9, 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 2 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-18T01:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:40.457 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:40.459 INFO:tasks.thrashosds.thrasher:j is {'epoch': 118, 'fsid': '62b46b83-8ef2-466f-b28b-2d97f265422e', 'created': '2024-09-18T01:02:13.493841+0000', 'modified': '2024-09-18T01:04:39.286867+0000', 'last_up_change': '2024-09-18T01:02:24.290620+0000', 'last_in_change': '2024-09-18T01:02:48.274416+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-18T01:02:24.388064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-18T01:02:30.676199+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T01:02:40.390732+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '4563fb16-8745-4e75-b654-1e96403d91b9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 1626166637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 1626166637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 1626166637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 1626166637}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 1626166637}]}, 'public_addr': '172.21.15.31:6817/1626166637', 'cluster_addr': '172.21.15.31:6819/1626166637', 'heartbeat_back_addr': '172.21.15.31:6823/1626166637', 'heartbeat_front_addr': '172.21.15.31:6821/1626166637', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '634965cb-d444-4721-b34a-afbc49105cab', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 2210970259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 2210970259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 2210970259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 2210970259}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 2210970259}]}, 'public_addr': '172.21.15.31:6801/2210970259', 'cluster_addr': '172.21.15.31:6803/2210970259', 'heartbeat_back_addr': '172.21.15.31:6807/2210970259', 'heartbeat_front_addr': '172.21.15.31:6805/2210970259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '83443eab-2fa4-4c7e-a459-edb1bb0ed8d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 617133675}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 617133675}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 617133675}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 617133675}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 617133675}]}, 'public_addr': '172.21.15.31:6809/617133675', 'cluster_addr': '172.21.15.31:6811/617133675', 'heartbeat_back_addr': '172.21.15.31:6815/617133675', 'heartbeat_front_addr': '172.21.15.31:6813/617133675', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5797982a-71c1-4d6a-9604-00fbc6bd16d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6808', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6809', 'nonce': 151031464}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6810', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6811', 'nonce': 151031464}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6814', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6815', 'nonce': 151031464}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6812', 'nonce': 151031464}, {'type': 'v1', 'addr': '172.21.15.98:6813', 'nonce': 151031464}]}, 'public_addr': '172.21.15.98:6809/151031464', 'cluster_addr': '172.21.15.98:6811/151031464', 'heartbeat_back_addr': '172.21.15.98:6815/151031464', 'heartbeat_front_addr': '172.21.15.98:6813/151031464', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f79e3329-5481-4f33-8d28-6386af186492', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6816', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6817', 'nonce': 1499036965}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6818', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6819', 'nonce': 1499036965}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6822', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6823', 'nonce': 1499036965}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6820', 'nonce': 1499036965}, {'type': 'v1', 'addr': '172.21.15.98:6821', 'nonce': 1499036965}]}, 'public_addr': '172.21.15.98:6817/1499036965', 'cluster_addr': '172.21.15.98:6819/1499036965', 'heartbeat_back_addr': '172.21.15.98:6823/1499036965', 'heartbeat_front_addr': '172.21.15.98:6821/1499036965', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8c61d2ec-b111-4783-a016-336ed33a0694', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6800', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6801', 'nonce': 2310498524}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6802', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6803', 'nonce': 2310498524}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6806', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6807', 'nonce': 2310498524}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6804', 'nonce': 2310498524}, {'type': 'v1', 'addr': '172.21.15.98:6805', 'nonce': 2310498524}]}, 'public_addr': '172.21.15.98:6801/2310498524', 'cluster_addr': '172.21.15.98:6803/2310498524', 'heartbeat_back_addr': '172.21.15.98:6807/2310498524', 'heartbeat_front_addr': '172.21.15.98:6805/2310498524', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.877700+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.360930+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.403205+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.952123+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T01:02:22.099545+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T01:02:21.694132+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.9', 'osds': [1, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.4', 'osds': [1]}, {'pgid': '3.7', 'osds': [1]}, {'pgid': '3.17', 'osds': [0]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 9, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T01:04:40.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:04:40.460 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:40.457+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:40.487 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:40.487 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:40.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:40.559 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:40.558+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:40.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:40.648 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:40.648 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:40.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:40.660 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:40.658+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:40.731 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:40.732 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:40.750 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:40.751 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:40.761 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:40.760+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:40.767 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:40.781 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:40.781 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:40.784 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:04:40.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:40.801 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-18T01:04:40.389822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":199991647,"num_objects":306,"num_object_clones":23,"num_object_copies":612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":195,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":194,"num_read":264245,"num_read_kb":1199316,"num_write":22795,"num_write_kb":288745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1100,"num_bytes_recovered":32025567,"num_keys_recovered":73,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20359,"ondisk_log_size":20359,"up":82,"acting":79,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":283132,"kb_used_data":9164,"kb_used_omap":7,"kb_used_meta":273848,"kb_avail":471576068,"statfs":{"total":483183820800,"available":482893893632,"internally_reserved":0,"allocated":9383936,"data_stored":7075909,"data_compressed":139127,"data_compressed_allocated":421888,"data_compressed_original":2682880,"omap_allocated":7973,"internal_metadata":280420571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9111711,"num_objects":-83,"num_object_clones":0,"num_object_copies":-166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":195,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":-83,"num_whiteouts":0,"num_read":361,"num_read_kb":313,"num_write":142,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":93831,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022976"},"pg_stats":[{"pgid":"3.1f","version":"115'649","reported_seq":10457,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292803+0000","last_change":"2024-09-18T01:04:36.374781+0000","last_active":"2024-09-18T01:04:39.292803+0000","last_peered":"2024-09-18T01:04:39.292803+0000","last_clean":"2024-09-18T01:04:39.292803+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:04:39.292803+0000","last_undegraded":"2024-09-18T01:04:39.292803+0000","last_fullsized":"2024-09-18T01:04:39.292803+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:38:29.869261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012022020000000001,"stat_sum":{"num_bytes":9612816,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":12632,"num_read_kb":58132,"num_write":538,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.1e","version":"114'1473","reported_seq":7662,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296510+0000","last_change":"2024-09-18T01:04:35.367542+0000","last_active":"2024-09-18T01:04:39.296510+0000","last_peered":"2024-09-18T01:04:39.296510+0000","last_clean":"2024-09-18T01:04:39.296510+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:04:39.296510+0000","last_undegraded":"2024-09-18T01:04:39.296510+0000","last_fullsized":"2024-09-18T01:04:39.296510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:01:35.951337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050279199999999995,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.1d","version":"114'449","reported_seq":7374,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296520+0000","last_change":"2024-09-18T01:04:35.375098+0000","last_active":"2024-09-18T01:04:39.296520+0000","last_peered":"2024-09-18T01:04:39.296520+0000","last_clean":"2024-09-18T01:04:39.296520+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:04:39.296520+0000","last_undegraded":"2024-09-18T01:04:39.296520+0000","last_fullsized":"2024-09-18T01:04:39.296520+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:05:51.104181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071168399999999997,"stat_sum":{"num_bytes":3276800,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":7621,"num_read_kb":29178,"num_write":431,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.1c","version":"114'837","reported_seq":12585,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296367+0000","last_change":"2024-09-18T01:04:35.374852+0000","last_active":"2024-09-18T01:04:39.296367+0000","last_peered":"2024-09-18T01:04:39.296367+0000","last_clean":"2024-09-18T01:04:39.296367+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:04:39.296367+0000","last_undegraded":"2024-09-18T01:04:39.296367+0000","last_fullsized":"2024-09-18T01:04:39.296367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":837,"log_dups_size":0,"ondisk_log_size":837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:56.872653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047605999999999999,"stat_sum":{"num_bytes":11495977,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":15632,"num_read_kb":43369,"num_write":706,"num_write_kb":8643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.1b","version":"116'622","reported_seq":4422,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.304521+0000","last_change":"2024-09-18T01:04:39.300648+0000","last_active":"2024-09-18T01:04:39.304521+0000","last_peered":"2024-09-18T01:04:39.304521+0000","last_clean":"2024-09-18T01:04:39.304521+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:04:39.304521+0000","last_undegraded":"2024-09-18T01:04:39.304521+0000","last_fullsized":"2024-09-18T01:04:39.304521+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:47:03.462411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087604600000000001,"stat_sum":{"num_bytes":8585313,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":5106,"num_read_kb":19572,"num_write":768,"num_write_kb":5929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.1a","version":"113'234","reported_seq":5934,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296435+0000","last_change":"2024-09-18T01:04:35.374685+0000","last_active":"2024-09-18T01:04:39.296435+0000","last_peered":"2024-09-18T01:04:39.296435+0000","last_clean":"2024-09-18T01:04:39.296435+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:04:39.296435+0000","last_undegraded":"2024-09-18T01:04:39.296435+0000","last_fullsized":"2024-09-18T01:04:39.296435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:15:42.778069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054461099999999997,"stat_sum":{"num_bytes":3346963,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":5413,"num_read_kb":19391,"num_write":230,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.19","version":"114'624","reported_seq":8555,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292855+0000","last_change":"2024-09-18T01:04:35.374535+0000","last_active":"2024-09-18T01:04:39.292855+0000","last_peered":"2024-09-18T01:04:39.292855+0000","last_clean":"2024-09-18T01:04:39.292855+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:04:39.292855+0000","last_undegraded":"2024-09-18T01:04:39.292855+0000","last_fullsized":"2024-09-18T01:04:39.292855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:30:21.931451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00111075,"stat_sum":{"num_bytes":41,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":8,"num_read":10840,"num_read_kb":39965,"num_write":578,"num_write_kb":9910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.18","version":"116'735","reported_seq":8749,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.303855+0000","last_change":"2024-09-18T01:04:39.303855+0000","last_active":"2024-09-18T01:04:39.303855+0000","last_peered":"2024-09-18T01:04:39.303855+0000","last_clean":"2024-09-18T01:04:39.303855+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:04:39.303855+0000","last_undegraded":"2024-09-18T01:04:39.303855+0000","last_fullsized":"2024-09-18T01:04:39.303855+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:20:22.292881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050874000000000004,"stat_sum":{"num_bytes":3426304,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":9266,"num_read_kb":83025,"num_write":702,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.17","version":"114'896","reported_seq":8200,"reported_epoch":118,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-18T01:04:39.316184+0000","last_change":"2024-09-18T01:04:39.316402+0000","last_active":"2024-09-18T01:04:39.316184+0000","last_peered":"2024-09-18T01:04:39.316184+0000","last_clean":"2024-09-18T01:04:36.369839+0000","last_became_active":"2024-09-18T01:04:39.315035+0000","last_became_peered":"2024-09-18T01:04:39.315035+0000","last_unstale":"2024-09-18T01:04:39.316184+0000","last_undegraded":"2024-09-18T01:04:39.295411+0000","last_fullsized":"2024-09-18T01:04:39.292985+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011695939999999999,"stat_sum":{"num_bytes":16360484,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":11,"num_read":10622,"num_read_kb":32379,"num_write":1047,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1702477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":16}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.16","version":"114'359","reported_seq":3795,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297585+0000","last_change":"2024-09-18T01:04:35.369226+0000","last_active":"2024-09-18T01:04:39.297585+0000","last_peered":"2024-09-18T01:04:39.297585+0000","last_clean":"2024-09-18T01:04:39.297585+0000","last_became_active":"2024-09-18T01:03:38.521858+0000","last_became_peered":"2024-09-18T01:03:38.521858+0000","last_unstale":"2024-09-18T01:04:39.297585+0000","last_undegraded":"2024-09-18T01:04:39.297585+0000","last_fullsized":"2024-09-18T01:04:39.297585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:19:37.892305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00131504,"stat_sum":{"num_bytes":3856993,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":3213,"num_read_kb":32432,"num_write":409,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.15","version":"114'583","reported_seq":9818,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297489+0000","last_change":"2024-09-18T01:04:35.368573+0000","last_active":"2024-09-18T01:04:39.297489+0000","last_peered":"2024-09-18T01:04:39.297489+0000","last_clean":"2024-09-18T01:04:39.297489+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:04:39.297489+0000","last_undegraded":"2024-09-18T01:04:39.297489+0000","last_fullsized":"2024-09-18T01:04:39.297489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:02:36.851040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010707259999999999,"stat_sum":{"num_bytes":9093211,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":11665,"num_read_kb":41433,"num_write":449,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.14","version":"114'575","reported_seq":5258,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.298022+0000","last_change":"2024-09-18T01:04:35.368603+0000","last_active":"2024-09-18T01:04:39.298022+0000","last_peered":"2024-09-18T01:04:39.298022+0000","last_clean":"2024-09-18T01:04:39.298022+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:04:39.298022+0000","last_undegraded":"2024-09-18T01:04:39.298022+0000","last_fullsized":"2024-09-18T01:04:39.298022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:33.038202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011586420000000001,"stat_sum":{"num_bytes":3178512,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":6144,"num_read_kb":17373,"num_write":608,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":219,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297693+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:04:39.297693+0000","last_peered":"2024-09-18T01:04:39.297693+0000","last_clean":"2024-09-18T01:04:39.297693+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:04:39.297693+0000","last_undegraded":"2024-09-18T01:04:39.297693+0000","last_fullsized":"2024-09-18T01:04:39.297693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"114'761","reported_seq":7062,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.293498+0000","last_change":"2024-09-18T01:04:35.368667+0000","last_active":"2024-09-18T01:04:39.293498+0000","last_peered":"2024-09-18T01:04:39.293498+0000","last_clean":"2024-09-18T01:04:39.293498+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:04:39.293498+0000","last_undegraded":"2024-09-18T01:04:39.293498+0000","last_fullsized":"2024-09-18T01:04:39.293498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:20:54.956202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010360409999999999,"stat_sum":{"num_bytes":18302976,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":971,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":219,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.293348+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:04:39.293348+0000","last_peered":"2024-09-18T01:04:39.293348+0000","last_clean":"2024-09-18T01:04:39.293348+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:04:39.293348+0000","last_undegraded":"2024-09-18T01:04:39.293348+0000","last_fullsized":"2024-09-18T01:04:39.293348+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"118'2109","reported_seq":10843,"reported_epoch":118,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-18T01:04:39.356517+0000","last_change":"2024-09-18T01:04:39.346487+0000","last_active":"2024-09-18T01:04:39.356517+0000","last_peered":"2024-09-18T01:04:39.356517+0000","last_clean":"2024-09-18T01:04:36.369415+0000","last_became_active":"2024-09-18T01:04:39.343591+0000","last_became_peered":"2024-09-18T01:04:39.343591+0000","last_unstale":"2024-09-18T01:04:39.356517+0000","last_undegraded":"2024-09-18T01:04:39.299486+0000","last_fullsized":"2024-09-18T01:04:39.293507+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2109,"log_dups_size":0,"ondisk_log_size":2109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019444239999999999,"stat_sum":{"num_bytes":1224192,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":90,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3360,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":2222926,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[1],"avail_no_missing":["1","4"],"object_location_counts":[{"shards":"1","objects":-1},{"shards":"1,4","objects":8}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":220,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297695+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:04:39.297695+0000","last_peered":"2024-09-18T01:04:39.297695+0000","last_clean":"2024-09-18T01:04:39.297695+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:04:39.297695+0000","last_undegraded":"2024-09-18T01:04:39.297695+0000","last_fullsized":"2024-09-18T01:04:39.297695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"114'461","reported_seq":6047,"reported_epoch":118,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-18T01:04:39.370376+0000","last_change":"2024-09-18T01:04:39.313517+0000","last_active":"2024-09-18T01:04:39.370376+0000","last_peered":"2024-09-18T01:04:39.370376+0000","last_clean":"2024-09-18T01:04:36.368951+0000","last_became_active":"2024-09-18T01:04:39.310329+0000","last_became_peered":"2024-09-18T01:04:39.310329+0000","last_unstale":"2024-09-18T01:04:39.370376+0000","last_undegraded":"2024-09-18T01:04:39.370376+0000","last_fullsized":"2024-09-18T01:04:39.293301+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458556,"stat_sum":{"num_bytes":9198592,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":5793,"num_read_kb":21958,"num_write":481,"num_write_kb":7398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":293972,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1","objects":4},{"shards":"1,3","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":220,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.293875+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:04:39.293875+0000","last_peered":"2024-09-18T01:04:39.293875+0000","last_clean":"2024-09-18T01:04:39.293875+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:04:39.293875+0000","last_undegraded":"2024-09-18T01:04:39.293875+0000","last_fullsized":"2024-09-18T01:04:39.293875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"114'360","reported_seq":8085,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296651+0000","last_change":"2024-09-18T01:04:35.368000+0000","last_active":"2024-09-18T01:04:39.296651+0000","last_peered":"2024-09-18T01:04:39.296651+0000","last_clean":"2024-09-18T01:04:39.296651+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:04:39.296651+0000","last_undegraded":"2024-09-18T01:04:39.296651+0000","last_fullsized":"2024-09-18T01:04:39.296651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:24.350783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086195499999999997,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7493,"num_read_kb":41289,"num_write":362,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.3","version":"114'1033","reported_seq":7309,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296540+0000","last_change":"2024-09-18T01:04:35.375424+0000","last_active":"2024-09-18T01:04:39.296540+0000","last_peered":"2024-09-18T01:04:39.296540+0000","last_clean":"2024-09-18T01:04:39.296540+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:04:39.296540+0000","last_undegraded":"2024-09-18T01:04:39.296540+0000","last_fullsized":"2024-09-18T01:04:39.296540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1033,"log_dups_size":0,"ondisk_log_size":1033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:51:44.106220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047023699999999998,"stat_sum":{"num_bytes":262144,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":9678,"num_read_kb":61479,"num_write":1307,"num_write_kb":7920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.2","version":"18'2","reported_seq":211,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292972+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:04:39.292972+0000","last_peered":"2024-09-18T01:04:39.292972+0000","last_clean":"2024-09-18T01:04:39.292972+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:04:39.292972+0000","last_undegraded":"2024-09-18T01:04:39.292972+0000","last_fullsized":"2024-09-18T01:04:39.292972+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"113'296","reported_seq":6649,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292882+0000","last_change":"2024-09-18T01:04:35.374222+0000","last_active":"2024-09-18T01:04:39.292882+0000","last_peered":"2024-09-18T01:04:39.292882+0000","last_clean":"2024-09-18T01:04:39.292882+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:04:39.292882+0000","last_undegraded":"2024-09-18T01:04:39.292882+0000","last_fullsized":"2024-09-18T01:04:39.292882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:42:44.603294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010174240000000001,"stat_sum":{"num_bytes":25061904,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":6144,"num_read_kb":49428,"num_write":289,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":220,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296406+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:04:39.296406+0000","last_peered":"2024-09-18T01:04:39.296406+0000","last_clean":"2024-09-18T01:04:39.296406+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:04:39.296406+0000","last_undegraded":"2024-09-18T01:04:39.296406+0000","last_fullsized":"2024-09-18T01:04:39.296406+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"114'373","reported_seq":4506,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292584+0000","last_change":"2024-09-18T01:04:35.368627+0000","last_active":"2024-09-18T01:04:39.292584+0000","last_peered":"2024-09-18T01:04:39.292584+0000","last_clean":"2024-09-18T01:04:39.292584+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:04:39.292584+0000","last_undegraded":"2024-09-18T01:04:39.292584+0000","last_fullsized":"2024-09-18T01:04:39.292584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:57.578625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011401689999999999,"stat_sum":{"num_bytes":1561600,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":315,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":220,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297155+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:04:39.297155+0000","last_peered":"2024-09-18T01:04:39.297155+0000","last_clean":"2024-09-18T01:04:39.297155+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:04:39.297155+0000","last_undegraded":"2024-09-18T01:04:39.297155+0000","last_fullsized":"2024-09-18T01:04:39.297155+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'461","reported_seq":6841,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292870+0000","last_change":"2024-09-18T01:04:35.373281+0000","last_active":"2024-09-18T01:04:39.292870+0000","last_peered":"2024-09-18T01:04:39.292870+0000","last_clean":"2024-09-18T01:04:39.292870+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:04:39.292870+0000","last_undegraded":"2024-09-18T01:04:39.292870+0000","last_fullsized":"2024-09-18T01:04:39.292870+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:20:58.983651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095606300000000005,"stat_sum":{"num_bytes":1179721,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":7261,"num_read_kb":41803,"num_write":419,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"2.3","version":"16'1","reported_seq":197,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296456+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:04:39.296456+0000","last_peered":"2024-09-18T01:04:39.296456+0000","last_clean":"2024-09-18T01:04:39.296456+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:04:39.296456+0000","last_undegraded":"2024-09-18T01:04:39.296456+0000","last_fullsized":"2024-09-18T01:04:39.296456+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":372,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297182+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:04:39.297182+0000","last_peered":"2024-09-18T01:04:39.297182+0000","last_clean":"2024-09-18T01:04:39.297182+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:04:39.297182+0000","last_undegraded":"2024-09-18T01:04:39.297182+0000","last_fullsized":"2024-09-18T01:04:39.297182+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"114'656","reported_seq":8273,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297040+0000","last_change":"2024-09-18T01:04:35.368104+0000","last_active":"2024-09-18T01:04:39.297040+0000","last_peered":"2024-09-18T01:04:39.297040+0000","last_clean":"2024-09-18T01:04:39.297040+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:04:39.297040+0000","last_undegraded":"2024-09-18T01:04:39.297040+0000","last_fullsized":"2024-09-18T01:04:39.297040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:59.242270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041992800000000002,"stat_sum":{"num_bytes":11548761,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":8662,"num_read_kb":60970,"num_write":843,"num_write_kb":15561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.9","version":"114'442","reported_seq":5024,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292725+0000","last_change":"2024-09-18T01:04:35.374106+0000","last_active":"2024-09-18T01:04:39.292725+0000","last_peered":"2024-09-18T01:04:39.292725+0000","last_clean":"2024-09-18T01:04:39.292725+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:04:39.292725+0000","last_undegraded":"2024-09-18T01:04:39.292725+0000","last_fullsized":"2024-09-18T01:04:39.292725+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:36.243155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080764300000000003,"stat_sum":{"num_bytes":1265664,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.a","version":"114'747","reported_seq":5350,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297402+0000","last_change":"2024-09-18T01:04:35.368269+0000","last_active":"2024-09-18T01:04:39.297402+0000","last_peered":"2024-09-18T01:04:39.297402+0000","last_clean":"2024-09-18T01:04:39.297402+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:04:39.297402+0000","last_undegraded":"2024-09-18T01:04:39.297402+0000","last_fullsized":"2024-09-18T01:04:39.297402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:09.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098971800000000011,"stat_sum":{"num_bytes":2964515,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":6687,"num_read_kb":22821,"num_write":914,"num_write_kb":4908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.b","version":"114'511","reported_seq":6885,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.293088+0000","last_change":"2024-09-18T01:04:35.373261+0000","last_active":"2024-09-18T01:04:39.293088+0000","last_peered":"2024-09-18T01:04:39.293088+0000","last_clean":"2024-09-18T01:04:39.293088+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:04:39.293088+0000","last_undegraded":"2024-09-18T01:04:39.293088+0000","last_fullsized":"2024-09-18T01:04:39.293088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:51:51.509370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018045369999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.c","version":"114'371","reported_seq":5990,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292726+0000","last_change":"2024-09-18T01:04:35.368206+0000","last_active":"2024-09-18T01:04:39.292726+0000","last_peered":"2024-09-18T01:04:39.292726+0000","last_clean":"2024-09-18T01:04:39.292726+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:04:39.292726+0000","last_undegraded":"2024-09-18T01:04:39.292726+0000","last_fullsized":"2024-09-18T01:04:39.292726+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:35:37.950947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083700599999999995,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.d","version":"114'526","reported_seq":6122,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297544+0000","last_change":"2024-09-18T01:04:35.368925+0000","last_active":"2024-09-18T01:04:39.297544+0000","last_peered":"2024-09-18T01:04:39.297544+0000","last_clean":"2024-09-18T01:04:39.297544+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:04:39.297544+0000","last_undegraded":"2024-09-18T01:04:39.297544+0000","last_fullsized":"2024-09-18T01:04:39.297544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:27.368878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012197099999999999,"stat_sum":{"num_bytes":4173312,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":6693,"num_read_kb":28822,"num_write":533,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.e","version":"114'397","reported_seq":6807,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.296437+0000","last_change":"2024-09-18T01:04:35.375213+0000","last_active":"2024-09-18T01:04:39.296437+0000","last_peered":"2024-09-18T01:04:39.296437+0000","last_clean":"2024-09-18T01:04:39.296437+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:04:39.296437+0000","last_undegraded":"2024-09-18T01:04:39.296437+0000","last_fullsized":"2024-09-18T01:04:39.296437+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:30.671680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067248000000000004,"stat_sum":{"num_bytes":13544960,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":340,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.f","version":"118'610","reported_seq":13183,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.364412+0000","last_change":"2024-09-18T01:04:35.368907+0000","last_active":"2024-09-18T01:04:39.364412+0000","last_peered":"2024-09-18T01:04:39.364412+0000","last_clean":"2024-09-18T01:04:39.364412+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:04:39.364412+0000","last_undegraded":"2024-09-18T01:04:39.364412+0000","last_fullsized":"2024-09-18T01:04:39.364412+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:30.819341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":2483712,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.10","version":"116'435","reported_seq":4545,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.301093+0000","last_change":"2024-09-18T01:04:39.301004+0000","last_active":"2024-09-18T01:04:39.301093+0000","last_peered":"2024-09-18T01:04:39.301093+0000","last_clean":"2024-09-18T01:04:39.301093+0000","last_became_active":"2024-09-18T01:04:39.295254+0000","last_became_peered":"2024-09-18T01:04:39.295254+0000","last_unstale":"2024-09-18T01:04:39.301093+0000","last_undegraded":"2024-09-18T01:04:39.301093+0000","last_fullsized":"2024-09-18T01:04:39.301093+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:34:57.084383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003691083,"stat_sum":{"num_bytes":2504742,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":5012,"num_read_kb":25885,"num_write":430,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.11","version":"114'385","reported_seq":4338,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297193+0000","last_change":"2024-09-18T01:04:35.367797+0000","last_active":"2024-09-18T01:04:39.297193+0000","last_peered":"2024-09-18T01:04:39.297193+0000","last_clean":"2024-09-18T01:04:39.297193+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:04:39.297193+0000","last_undegraded":"2024-09-18T01:04:39.297193+0000","last_fullsized":"2024-09-18T01:04:39.297193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:30:54.231531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062143300000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":4079,"num_read_kb":19488,"num_write":370,"num_write_kb":3472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.12","version":"114'962","reported_seq":10395,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.292585+0000","last_change":"2024-09-18T01:04:35.368204+0000","last_active":"2024-09-18T01:04:39.292585+0000","last_peered":"2024-09-18T01:04:39.292585+0000","last_clean":"2024-09-18T01:04:39.292585+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:04:39.292585+0000","last_undegraded":"2024-09-18T01:04:39.292585+0000","last_fullsized":"2024-09-18T01:04:39.292585+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:33:56.508957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095268399999999997,"stat_sum":{"num_bytes":21929563,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":12763,"num_read_kb":52311,"num_write":1220,"num_write_kb":15851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"2"}]},{"pgid":"3.13","version":"114'307","reported_seq":3848,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-18T01:04:39.297293+0000","last_change":"2024-09-18T01:04:35.368272+0000","last_active":"2024-09-18T01:04:39.297293+0000","last_peered":"2024-09-18T01:04:39.297293+0000","last_clean":"2024-09-18T01:04:39.297293+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:04:39.297293+0000","last_undegraded":"2024-09-18T01:04:39.297293+0000","last_fullsized":"2024-09-18T01:04:39.297293+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:28.987621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000862246,"stat_sum":{"num_bytes":9961472,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":3257,"num_read_kb":13085,"num_write":293,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":199401260,"num_objects":302,"num_object_clones":23,"num_object_copies":604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":195,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":194,"num_read":264128,"num_read_kb":1199215,"num_write":22634,"num_write_kb":286327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":32025548,"num_keys_recovered":73,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2240512,"data_stored":2985786,"data_compressed":121995,"data_compressed_allocated":348160,"data_compressed_original":1527808,"omap_allocated":0,"internal_metadata":0},"log_size":20239,"ondisk_log_size":20239,"up":64,"acting":61,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640285,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640287,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46848,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324992,"statfs":{"total":96636764160,"available":96588791808,"internally_reserved":0,"allocated":1814528,"data_stored":1157387,"data_compressed":1438,"data_compressed_allocated":40960,"data_compressed_original":278528,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47060,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324780,"statfs":{"total":96636764160,"available":96588574720,"internally_reserved":0,"allocated":2023424,"data_stored":1885504,"data_compressed":67346,"data_compressed_allocated":143360,"data_compressed_original":921600,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77176,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294664,"statfs":{"total":96636764160,"available":96557735936,"internally_reserved":0,"allocated":1609728,"data_stored":791979,"data_compressed":511,"data_compressed_allocated":20480,"data_compressed_original":86016,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65088,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306752,"statfs":{"total":96636764160,"available":96570114048,"internally_reserved":0,"allocated":2007040,"data_stored":1322399,"data_compressed":1872,"data_compressed_allocated":65536,"data_compressed_original":335872,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640286,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46960,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324880,"statfs":{"total":96636764160,"available":96588677120,"internally_reserved":0,"allocated":1929216,"data_stored":1918640,"data_compressed":67960,"data_compressed_allocated":151552,"data_compressed_original":1060864,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":479232,"data_stored":756593,"data_compressed":59410,"data_compressed_allocated":114688,"data_compressed_original":487424,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":700318,"data_compressed":1725,"data_compressed_allocated":61440,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":228473,"data_compressed":626,"data_compressed_allocated":24576,"data_compressed_original":106496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":714675,"data_compressed":58796,"data_compressed_allocated":106496,"data_compressed_original":348160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":585727,"data_compressed":1438,"data_compressed_allocated":40960,"data_compressed_original":278528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:04:40.804 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [4, 3, 0, 5] 2024-09-18T01:04:40.804 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.2', '4', '3', '0', '5'] 2024-09-18T01:04:40.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.2 4 3 0 5 2024-09-18T01:04:40.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:40.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:40.820 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:40.820 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:40.862 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:40.860+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:40.900 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:40.900 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:40.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:40.963 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:40.962+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 112.741895169, 2024-09-18T01:04:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 112.737269266, 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 112.737191217, 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.680395174, 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.68036931, 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.68034683, 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.680325794, 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.6802984, 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.680274095, 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.67678803299999, 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.676767567, 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.676726227, 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.676706189, 2024-09-18T01:04:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.676663858, 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.67664286599999, 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.67659189600001, 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.444451508, 2024-09-18T01:04:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:24948 3.9 3:945f9464:::rbd_data.1103102c66ed.000000000000015d:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.570783878, 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89682947899999998, 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 24948 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448239+0000", 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1579999999999998e-06 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448250+0000", 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0061000000000001e-05 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448256+0000", 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6699999999999997e-06 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448310+0000", 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3888999999999999e-05 2024-09-18T01:04:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448321+0000", 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1245e-05 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.342977+0000", 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89465616699999995 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343091+0000", 2024-09-18T01:04:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011382099999999999 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343168+0000", 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7015999999999996e-05 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343923+0000", 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075445099999999995 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344936+0000", 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010128820000000001 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345011+0000", 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5519999999999995e-05 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345064+0000", 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2913e-05 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:24951 3.9 3:90848b56:::rbd_data.1103102c66ed.00000000000001c1:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.570608119999999, 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89740894000000004, 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 24951 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448411+0000", 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6400000000000002e-07 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448415+0000", 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-06 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448422+0000", 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-18T01:04:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448427+0000", 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.313e-06 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448481+0000", 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4604000000000001e-05 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448492+0000", 2024-09-18T01:04:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344058+0000", 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89556604799999995 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344133+0000", 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000006e-05 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344202+0000", 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589000000000001e-05 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345247+0000", 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010448110000000001 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345693+0000", 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000446203 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345754+0000", 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0445999999999997e-05 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345819+0000", 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5568999999999995e-05 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:26172 3.10 3:0a0d7897:::rbd_data.1103ca939bf9.000000000000007f:head [delete] snapc 9=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e115)", 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:36.366948+0000", 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6520702719999996, 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0286951040000001, 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 26172 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366948+0000", 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366948+0000", 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366950+0000", 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.716e-06 2024-09-18T01:04:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366957+0000", 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366970+0000", 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2814e-05 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366981+0000", 2024-09-18T01:04:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.151e-05 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.367048+0000", 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7156999999999994e-05 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.367056+0000", 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2199999999999992e-06 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.368937+0000", 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001880187 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.368948+0000", 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1223000000000001e-05 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.393596+0000", 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.024648378 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.393759+0000", 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163108 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.393910+0000", 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015027700000000001 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.394822+0000", 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091247099999999999 2024-09-18T01:04:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.395467+0000", 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064444000000000003 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.395530+0000", 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3250000000000006e-05 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.395643+0000", 2024-09-18T01:04:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011353600000000001 2024-09-18T01:04:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.132 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:41.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:41.134 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:41.132+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:41.234 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:41.233+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:41.282 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:41.282 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:41.282 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:41.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:41.334 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:41.333+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:41.390 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-18T01:04:41.393 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.2 pg_upmap_items mapping to [4->3,0->5] 2024-09-18T01:04:41.405 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:41.405 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:41.406 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:41.406 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:41.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:41.436 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:41.435+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:41.537 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:41.535+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:41.571 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.834367774, 2024-09-18T01:04:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.834347816000005, 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:41.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.834328874999997, 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:41.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:41.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.834307366000004, 2024-09-18T01:04:41.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:41.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:41.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.834281637999993, 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:41.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:41.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825527065000003, 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:41.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825507935000005, 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:41.611 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:41.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825489089000001, 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:41.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:41.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:41.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825470210999995, 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825448346000002, 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:41.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:41.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:41.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825424710999997, 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:41.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:41.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825391061000005, 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825363206000006, 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:41.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:41.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825343383000003, 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:41.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825322948999997, 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:41.661 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:41.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:41.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:41.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825300869000003, 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:41.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:41.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825278546999996, 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:41.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:41.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:41.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825256151000005, 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:41.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:41.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:41.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:41.687 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.825235731999996, 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:41.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:41.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:26193 3.7 3.4ec2ea07 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:37.401191+0000", 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1674233090000001, 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.88800681800000003, 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:41.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 26193 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401191+0000", 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401191+0000", 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401192+0000", 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1319999999999999e-06 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401198+0000", 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8660000000000002e-06 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401209+0000", 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1341e-05 2024-09-18T01:04:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401217+0000", 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5689999999999999e-06 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401296+0000", 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9853999999999998e-05 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401308+0000", 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2048999999999999e-05 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:38.289186+0000", 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.88787759700000002 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:38.289197+0000", 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1409999999999999e-05 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:41.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:41.699 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:41.700 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:41.701 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:41.699+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:41.801 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:41.799+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:41.814 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:41.814 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:41.815 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:41.815 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:41.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:41.902 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:41.901+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:41.926 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:41.926 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:41.926 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:41.926 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:41.927 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:41.935 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:42.003 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:42.001+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:42.079 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:42.079 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:42.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.03361657000001, 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.080 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:04:42.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:04:42.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.084 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:04:42.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:04:42.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.085 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.033605419, 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:04:42.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:04:42.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:42.090 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.03359358900001, 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:42.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:04:42.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.03357690599999, 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:04:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.099 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.033565669, 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.100 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.102 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.032243564, 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.105 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.032230849, 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:04:42.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.111 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:04:42.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.114 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.032185865, 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.115 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:04:42.116 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:42.119 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.032158241, 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.120 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.121 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:04:42.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.02861086199999, 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.124 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:42.125 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:04:42.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.028250755, 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.129 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.130 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:04:42.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.028224642, 2024-09-18T01:04:42.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:42.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.137 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.026045254, 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.138 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:04:42.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:04:42.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:42.142 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.026035459, 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:42.143 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.145 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:04:42.146 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.02602468000001, 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.147 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:04:42.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.026004884, 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.152 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:04:42.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.024194526, 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:04:42.156 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:04:42.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 100.024183978, 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.161 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:42.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:04:42.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 64.552257101999999, 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.169 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:04:42.170 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:26149 3.1b 3:d8d208d4:::rbd_data.1103ca939bf9.0000000000000020:head [delete] snapc 9=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e115)", 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:04:36.366973+0000", 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 5.7102604699999997, 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0278071440000001, 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.171 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 26149 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366973+0000", 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366973+0000", 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366975+0000", 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2189999999999998e-06 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366984+0000", 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7690000000000002e-06 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367005+0000", 2024-09-18T01:04:42.173 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1747e-05 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367019+0000", 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3617e-05 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367101+0000", 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2379000000000005e-05 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.174 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367110+0000", 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.884e-06 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.370160+0000", 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0030499640000000001 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.370168+0000", 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2470000000000002e-06 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.175 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.392709+0000", 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0225411740000001 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.392834+0000", 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000125034 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0,4", 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.392949+0000", 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000114799 2024-09-18T01:04:42.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.393531+0000", 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00058285699999999999 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.393963+0000", 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00043142799999999998 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.394674+0000", 2024-09-18T01:04:42.177 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00071089199999999997 2024-09-18T01:04:42.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.178 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.178 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:42.178 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.394717+0000", 2024-09-18T01:04:42.178 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3754999999999997e-05 2024-09-18T01:04:42.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.178 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.179 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.394780+0000", 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2378999999999993e-05 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.180 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:42.182 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:42.182 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:42.180+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:42.282 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:42.281+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:42.326 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:42.326 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:42.326 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:42.326 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:42.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:42.384 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:42.381+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:42.483 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:42.482+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:42.487 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:42.488 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:42.488 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:42.498 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:42.584 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:42.583+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:42.642 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:42.642 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:42.642 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:42.642 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.344701142000005, 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.643 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.644 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:42.645 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.646 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.344677024999996, 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.647 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.648 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:42.649 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.344597785000005, 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.652 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:42.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.344561565999996, 2024-09-18T01:04:42.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:42.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.344532952999998, 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:42.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:42.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.344512088000002, 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.665 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:42.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:42.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.669 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343894371000005, 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.670 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:42.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:42.673 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343865104000002, 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.674 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.675 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:42.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.678 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343798813000006, 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:42.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.682 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343678045999994, 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:42.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:42.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343459600000003, 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:42.687 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:42.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343438238999994, 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.692 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:42.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.695 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343284299999993, 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.696 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343264582000003, 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.700 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:42.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:42.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:42.704 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343244272000007, 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:42.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.708 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343224386000003, 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.709 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:42.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343204427000003, 2024-09-18T01:04:42.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:42.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:42.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343184429000004, 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:42.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.721 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343164762000001, 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.722 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 96.343144859999995, 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:42.726 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:42.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:42.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:42.730 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:42.732 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:42.732 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:42.730+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:42.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T01:04:42.833 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:42.832+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:42.875 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:42.875 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:42.875 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:42.875 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:42.884 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:42.934 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:42.933+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:42.970 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:42.970 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:42.987 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:42.987 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:42.988 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:42.988 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:42.988 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:42.988 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:42.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:43.003 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:43.021 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:43.037 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:43.037 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:43.038 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:43.037+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:43.055 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:43.055 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:43.136 INFO:tasks.ceph.osd.4.smithi098.stderr:2024-09-18T01:04:43.135+0000 7f69411d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31710) UID: 0 2024-09-18T01:04:43.141 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:43.141 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:43.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:43.237 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:43.235+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:43.258 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:43.258 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:43.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:43.338 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:43.337+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:43.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T01:04:43.424 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:43.424 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:43.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:43.476 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:43.474+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:43.540 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:43.539+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:43.586 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:43.586 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:43.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:43.641 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:43.639+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:43.703 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:43.704 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:43.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:43.753 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:43.751+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:43.843 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:43.841+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:43.859 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:43.859 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:43.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:43.952 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:43.950+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:44.015 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.016 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:44.066 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:44.064+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:44.146 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:44.145+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:44.173 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.173 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:44.247 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:44.245+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:44.288 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.288 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:44.381 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:44.379+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:44.413 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-18T01:04:44.448 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:44.447+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:44.452 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.453 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.462 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:44.549 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:44.547+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:44.576 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.576 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.586 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:44.650 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:44.648+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:44.688 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.688 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.697 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:44.750 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:44.749+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:44.800 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.800 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.809 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:44.851 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:44.850+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:44.952 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:44.951+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:44.952 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:44.952 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:44.961 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:45.053 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:45.052+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:45.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T01:04:45.103 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:45.103 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:45.111 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:45.154 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:45.153+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:45.195 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-18T01:04:45.195 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T01:04:45.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-18T01:04:45.212 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T01:04:45.228 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-18T01:04:45.229 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T01:04:45.245 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-18T01:04:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T01:04:45.255 INFO:teuthology.orchestra.run.smithi098.stdout:ERROR: (22) Invalid argument 2024-09-18T01:04:45.255 INFO:teuthology.orchestra.run.smithi098.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T01:04:45.255 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:45.254+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:45.263 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-18T01:04:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T01:04:45.264 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:45.279 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-18T01:04:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T01:04:45.355 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:45.354+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:45.408 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:45.408 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:45.408 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:45.408 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:45.417 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:45.458 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:45.457+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:45.557 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:45.555+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:45.568 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:45.568 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:45.568 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:45.568 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:45.568 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:45.578 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:45.660 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:45.657+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:45.682 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:45.682 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:45.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:45.682 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:45.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 133.30615388000001, 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.683 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.684 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:45.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:45.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 132.30814257500001, 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:45.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:45.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:45.694 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.696 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 131.30046562300001, 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.697 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:45.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 122.202607673, 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:45.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:45.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:45.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:45.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.346526661, 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.711 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:45.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:45.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:45.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.346157365, 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.717 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:45.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.722 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.34441852, 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.723 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:45.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:45.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:45.727 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.34434663, 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:45.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.729 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.731 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.25648062, 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.732 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:45.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:45.737 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.250289361, 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:45.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.24875967600001, 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.743 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:45.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:45.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:45.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.24837998700001, 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.749 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:45.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.752 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:45.754 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.232881584, 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.755 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:45.756 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.757 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.758 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:45.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.760 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.232674825, 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.761 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.762 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:45.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.764 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:45.765 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.766 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:45.767 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.232257068, 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.768 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.769 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.770 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:45.771 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:45.772 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.23204783, 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.773 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.774 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:45.775 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.776 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.231983896, 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:45.777 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.778 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:45.779 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.780 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 118.23123569099999, 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.782 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.783 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:45.784 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.785 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:45.786 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 117.972296886, 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.787 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.788 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:45.789 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.790 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:45.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.792 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 117.971433268, 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:45.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:45.794 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.795 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:45.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:45.797 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.798 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.799 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.799 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:45.799 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:45.799 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:45.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T01:04:45.801 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-18T01:04:45.798+0000 7fb1830cd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31814) UID: 0 2024-09-18T01:04:45.901 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-18T01:04:45.899+0000 7fb4c893b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31817) UID: 0 2024-09-18T01:04:45.910 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:45.910 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:45.910 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:45.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T01:04:46.002 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:46.001+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:46.066 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:46.066 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:46.067 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:46.067 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:46.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T01:04:46.102 INFO:tasks.ceph.osd.5.smithi098.stderr:2024-09-18T01:04:46.102+0000 7f66c03ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31713) UID: 0 2024-09-18T01:04:46.183 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2016 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4275200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.863257097, 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028134598, 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2016 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318465+0000", 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318467+0000", 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0559999999999999e-06 2024-09-18T01:04:46.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318474+0000", 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6769999999999998e-06 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318479+0000", 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346428+0000", 2024-09-18T01:04:46.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027948853999999999 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346499+0000", 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1823999999999995e-05 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346599+0000", 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9846000000000006e-05 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2017 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4284416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.86323658800001, 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028308997999999998, 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2017 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318485+0000", 2024-09-18T01:04:46.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318486+0000", 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318487+0000", 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318494+0000", 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318498+0000", 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6299999999999997e-06 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346615+0000", 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028116727000000001 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346698+0000", 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2603999999999997e-05 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346794+0000", 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6413e-05 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2018 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4293632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.863216783, 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028464146999999999, 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2018 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318505+0000", 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318506+0000", 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-07 2024-09-18T01:04:46.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318515+0000", 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3260000000000002e-06 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318522+0000", 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318532+0000", 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8290000000000003e-06 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346819+0000", 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028286733000000001 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346889+0000", 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0263000000000001e-05 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346969+0000", 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0167999999999995e-05 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2019 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4302848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.863182143, 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028604193, 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2019 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318540+0000", 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318542+0000", 2024-09-18T01:04:46.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.165e-06 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318549+0000", 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7109999999999998e-06 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318554+0000", 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3809999999999998e-06 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.346997+0000", 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028442742999999999 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347063+0000", 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6011000000000002e-05 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347144+0000", 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0737999999999998e-05 2024-09-18T01:04:46.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2020 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4312064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.863160642, 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0287654, 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2020 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:46.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318561+0000", 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318563+0000", 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9439999999999999e-06 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318572+0000", 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5990000000000002e-06 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318577+0000", 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2820000000000004e-06 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347172+0000", 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028594519999999998 2024-09-18T01:04:46.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347239+0000", 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6885999999999996e-05 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347327+0000", 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7800000000000006e-05 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2021 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4321280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.86313746499999, 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029003260999999999, 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2021 2024-09-18T01:04:46.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318584+0000", 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318585+0000", 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318587+0000", 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318596+0000", 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0450000000000006e-06 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.318601+0000", 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347422+0000", 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028820077999999999 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347503+0000", 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1570000000000007e-05 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.347588+0000", 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4489999999999999e-05 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2238 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4220928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.859148633, 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028226439999999998, 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2238 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322573+0000", 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322574+0000", 2024-09-18T01:04:46.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322576+0000", 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322582+0000", 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322587+0000", 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999998e-06 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350637+0000", 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028050559999999999 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350704+0000", 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7271e-05 2024-09-18T01:04:46.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350800+0000", 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5159000000000002e-05 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2239 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4230144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.85912908, 2024-09-18T01:04:46.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028361556, 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2239 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322593+0000", 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322595+0000", 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322601+0000", 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322606+0000", 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9810000000000003e-06 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350830+0000", 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028223709999999999 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350894+0000", 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3528999999999994e-05 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350954+0000", 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0609000000000002e-05 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2240 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4239360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.859108576, 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028521309000000002, 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2240 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322613+0000", 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322614+0000", 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322616+0000", 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322622+0000", 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322627+0000", 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-18T01:04:46.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.350983+0000", 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028355642 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351047+0000", 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4733000000000005e-05 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351135+0000", 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7069000000000003e-05 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2241 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4248576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.859087712, 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029047238999999999, 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2241 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322634+0000", 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322637+0000", 2024-09-18T01:04:46.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322643+0000", 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6069999999999996e-06 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322648+0000", 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7330000000000003e-06 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351519+0000", 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028871463999999999 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351588+0000", 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8683999999999995e-05 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351681+0000", 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3300000000000005e-05 2024-09-18T01:04:46.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2242 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4257792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.85906781200001, 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029225153, 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2242 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:46.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322654+0000", 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322657+0000", 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322666+0000", 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1819999999999995e-06 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322671+0000", 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351710+0000", 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029039045999999999 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351787+0000", 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7515000000000006e-05 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351879+0000", 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1615999999999998e-05 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2243 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4267008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.859043724, 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029394650000000001, 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2243 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322678+0000", 2024-09-18T01:04:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322681+0000", 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2989999999999999e-06 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322688+0000", 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322700+0000", 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2563e-05 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351908+0000", 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029208206 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.351982+0000", 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3603999999999995e-05 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352073+0000", 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0729999999999994e-05 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2244 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4276224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.859012404, 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029606416999999999, 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2244 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322709+0000", 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322710+0000", 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9399999999999995e-07 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322729+0000", 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522999999999999e-05 2024-09-18T01:04:46.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322739+0000", 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6749999999999997e-06 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322745+0000", 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9000000000000003e-06 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352109+0000", 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029363842000000001 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352231+0000", 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121783 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352316+0000", 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5199999999999997e-05 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2245 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4285440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.85896779799999, 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029717691000000001, 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2245 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322754+0000", 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322757+0000", 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322766+0000", 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1269999999999992e-06 2024-09-18T01:04:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322771+0000", 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352345+0000", 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029574248000000001 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352397+0000", 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2126999999999997e-05 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352472+0000", 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4382999999999996e-05 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2246 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4294656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:46.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.85893700699999, 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029819546999999998, 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2246 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322785+0000", 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322788+0000", 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322796+0000", 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5439999999999999e-06 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322802+0000", 2024-09-18T01:04:46.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5759999999999996e-06 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352500+0000", 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029697840999999999 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352538+0000", 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8133999999999998e-05 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352604+0000", 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6667000000000004e-05 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2247 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4303872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.85891352199999, 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029909794, 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2247 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322808+0000", 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322809+0000", 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322811+0000", 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0219999999999999e-06 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322818+0000", 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7700000000000004e-06 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322830+0000", 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2245999999999999e-05 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352630+0000", 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029799781000000001 2024-09-18T01:04:46.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352666+0000", 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6177000000000003e-05 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352718+0000", 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2222999999999998e-05 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2248 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4313088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.858884467, 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030087624, 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2248 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.260 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322837+0000", 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322838+0000", 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322840+0000", 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322847+0000", 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7190000000000004e-06 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322852+0000", 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352746+0000", 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029894371999999999 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352835+0000", 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9042000000000006e-05 2024-09-18T01:04:46.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352925+0000", 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9949000000000002e-05 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:2249 3.12 3:496d5ad6:::rbd_data.110372aa4379.0000000000000003:7 [read 4322304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.858863708, 2024-09-18T01:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030236249, 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2249 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:46.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322858+0000", 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322861+0000", 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322867+0000", 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5169999999999996e-06 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.322872+0000", 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-18T01:04:46.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.352952+0000", 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030079763999999998 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353015+0000", 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2792999999999993e-05 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.353094+0000", 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9612999999999998e-05 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:484 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 12288~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.600922961000002, 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71588290099999996, 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 484 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580799+0000", 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580800+0000", 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6500000000000008e-07 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580805+0000", 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580815+0000", 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0943e-05 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580823+0000", 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8720000000000002e-06 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580870+0000", 2024-09-18T01:04:46.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6615e-05 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580880+0000", 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6709999999999994e-06 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296317+0000", 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71543726500000004 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296497+0000", 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180027 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296682+0000", 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000184799 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:487 3.17 3:eadc096d:::rbd_data.1103526f3c9c.0000000000000031:head [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.600839252, 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71623777600000005, 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 487 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580883+0000", 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:46.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580886+0000", 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.667e-06 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580893+0000", 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.580899+0000", 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8440000000000003e-06 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581006+0000", 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107159 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.581016+0000", 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0227999999999999e-05 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296868+0000", 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.71585158500000001 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.296960+0000", 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2578999999999996e-05 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.297120+0000", 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000159863 2024-09-18T01:04:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.280 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:46.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T01:04:46.282 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-18T01:04:46.280+0000 7fc6c5702640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31708) UID: 0 2024-09-18T01:04:46.382 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:46.380+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:46.389 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:46.390 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:46.390 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:46.390 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:46.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T01:04:46.417 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-18T01:04:46.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:04:46.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:04:46.485 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-18T01:04:46.483+0000 7fb3ab4fc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31821) UID: 0 2024-09-18T01:04:46.543 INFO:teuthology.orchestra.run.smithi164.stdout:All operations completed A-OK! 2024-09-18T01:04:46.553 DEBUG:teuthology.parallel:result is None 2024-09-18T01:04:46.553 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-18T01:04:46.565 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-18T01:04:46.579 INFO:tasks.thrashosds:joining thrashosds 2024-09-18T01:04:46.580 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:46.580 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:46.580 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:46.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T01:04:46.730 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:46.730 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:46.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:46.730 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:46.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:1075 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140656152831760 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.45088245399999, 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.021784502, 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277123+0000", 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277126+0000", 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.582e-06 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277135+0000", 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3989999999999993e-06 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277159+0000", 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3680999999999999e-05 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277172+0000", 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3281e-05 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277256+0000", 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4010000000000004e-05 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.277265+0000", 2024-09-18T01:04:46.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285171+0000", 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285185+0000", 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3753e-05 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.294779+0000", 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.009594485 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295074+0000", 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029467499999999999 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297071+0000", 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001997385 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297120+0000", 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8986999999999997e-05 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298908+0000", 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017873399999999999 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1140 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [watch reconnect cookie 140344096589968 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.446256551, 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017733725, 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1140 2024-09-18T01:04:46.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281749+0000", 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281751+0000", 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.95e-06 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281758+0000", 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4099999999999996e-06 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281782+0000", 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4221999999999999e-05 2024-09-18T01:04:46.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281794+0000", 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1796e-05 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281870+0000", 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6287999999999999e-05 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281878+0000", 2024-09-18T01:04:46.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2709999999999992e-06 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285646+0000", 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037682359999999999 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285657+0000", 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767999999999999e-05 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295357+0000", 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0096998370000001 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295419+0000", 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1634e-05 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297378+0000", 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019597709999999999 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297426+0000", 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7604e-05 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.299483+0000", 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020569379999999999 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4489.0:1141 3.2 3:409f0959:::rbd_header.1103b05c2cf1:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.446178502, 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018476562, 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4489", 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1141 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281827+0000", 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281828+0000", 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-07 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281833+0000", 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281842+0000", 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9549999999999998e-06 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281848+0000", 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2639999999999997e-06 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281902+0000", 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3819000000000001e-05 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.281905+0000", 2024-09-18T01:04:46.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285684+0000", 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00377867 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:48.285691+0000", 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3250000000000002e-06 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295594+0000", 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0099027860000001 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for degraded object", 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.295674+0000", 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0134000000000002e-05 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297662+0000", 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019880079999999999 2024-09-18T01:04:46.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.297743+0000", 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0723000000000006e-05 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298022+0000", 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027947099999999999 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:46.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.298918+0000", 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089542199999999999 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300219+0000", 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001301422 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300249+0000", 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0018e-05 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:49.300304+0000", 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4580999999999998e-05 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1294 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1783808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.389382459, 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998994999999998, 2024-09-18T01:04:46.753 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1294 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338623+0000", 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338624+0000", 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338626+0000", 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-18T01:04:46.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338636+0000", 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8220000000000002e-06 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338641+0000", 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4890000000000003e-06 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389538+0000", 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050896299999999998 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389579+0000", 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1703999999999999e-05 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389622+0000", 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3101e-05 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1296 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1793024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.389356595, 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051154403000000001, 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1296 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338649+0000", 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338650+0000", 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338652+0000", 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.063e-06 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338659+0000", 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7090000000000002e-06 2024-09-18T01:04:46.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338665+0000", 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1749999999999996e-06 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389655+0000", 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050990021000000003 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389731+0000", 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6752e-05 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389804+0000", 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2243999999999995e-05 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1297 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:46.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.389334115, 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051365194000000003, 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1297 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338672+0000", 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338674+0000", 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9549999999999999e-06 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338681+0000", 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0040000000000002e-06 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338686+0000", 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1089999999999997e-06 2024-09-18T01:04:46.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389848+0000", 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051161611000000003 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.389952+0000", 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010384 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390037+0000", 2024-09-18T01:04:46.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5297999999999995e-05 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1299 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1811456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.389313079, 2024-09-18T01:04:46.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051495314, 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1299 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:46.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338693+0000", 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338695+0000", 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338703+0000", 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689999999999994e-06 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338708+0000", 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2599999999999996e-06 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390057+0000", 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051348380999999998 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390130+0000", 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3730000000000001e-05 2024-09-18T01:04:46.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390188+0000", 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7550999999999998e-05 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.772 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1300 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.389285685, 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051568348999999999, 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1300 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338720+0000", 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338721+0000", 2024-09-18T01:04:46.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338724+0000", 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338732+0000", 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338737+0000", 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2510000000000002e-06 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390203+0000", 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051466158999999997 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390243+0000", 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9224999999999998e-05 2024-09-18T01:04:46.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390288+0000", 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5717999999999998e-05 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1302 3.0 3:02a34c9e:::rbd_data.1103a6f5bae0.0000000000000009:8 [read 1829888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.38926137999999, 2024-09-18T01:04:46.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051630028000000001, 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1302 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338744+0000", 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338745+0000", 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338747+0000", 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0190000000000001e-06 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338755+0000", 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8189999999999992e-06 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.338760+0000", 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0819999999999998e-06 2024-09-18T01:04:46.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390303+0000", 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051543494000000002 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390339+0000", 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6022999999999997e-05 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.390374+0000", 2024-09-18T01:04:46.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5018999999999998e-05 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1352 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.385775318, 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050943789000000003, 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:46.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1352 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342230+0000", 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342231+0000", 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342233+0000", 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342240+0000", 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342245+0000", 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393169+0000", 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050924247999999998 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393174+0000", 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1353 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.38575485200001, 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050958514000000003, 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1353 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342251+0000", 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342253+0000", 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9640000000000002e-06 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342259+0000", 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0270000000000001e-06 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342264+0000", 2024-09-18T01:04:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393201+0000", 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050937048999999998 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393209+0000", 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1355 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.385713512, 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050960990999999997, 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1355 2024-09-18T01:04:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342292+0000", 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342293+0000", 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342295+0000", 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.951e-06 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342301+0000", 2024-09-18T01:04:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9689999999999999e-06 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342305+0000", 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393248+0000", 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050942116000000003 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393253+0000", 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7309999999999998e-06 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1356 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.38569347399999, 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050987589999999999, 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1356 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342312+0000", 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342313+0000", 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342315+0000", 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9259999999999999e-06 2024-09-18T01:04:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342322+0000", 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342327+0000", 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393292+0000", 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050964210000000003 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393300+0000", 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.388e-06 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1358 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.385651143, 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050977313000000003, 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1358 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342355+0000", 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-09-18T01:04:46.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342357+0000", 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342364+0000", 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4860000000000002e-06 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342369+0000", 2024-09-18T01:04:46.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1660000000000002e-06 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393327+0000", 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050957939000000001 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393332+0000", 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1359 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.385630151, 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050983197000000001, 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1359 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342376+0000", 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342385+0000", 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0219999999999993e-06 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342392+0000", 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8120000000000002e-06 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342398+0000", 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9989999999999997e-06 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393350+0000", 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050952638000000001 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393359+0000", 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:1361 3.0 3.7932c540 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 108.385579181, 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050965796000000001, 2024-09-18T01:04:46.808 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1361 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:46.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.342427+0000", 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344117+0000", 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016903420000000001 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344122+0000", 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344131+0000", 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.799e-06 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.344138+0000", 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393385+0000", 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049247082999999997 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:02:58.393392+0000", 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5480000000000004e-06 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5364.0:456 3.7 3:e52c2f83:::rbd_object_map.1103526f3c9c:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.153438793000007, 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74483651399999995, 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5364", 2024-09-18T01:04:46.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1758960970", 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574567+0000", 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574568+0000", 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1599999999999999e-06 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574573+0000", 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574584+0000", 2024-09-18T01:04:46.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1464999999999999e-05 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574592+0000", 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.311e-06 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574637+0000", 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5759e-05 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:42.574645+0000", 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317018+0000", 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74237348400000003 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317415+0000", 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039666499999999998 2024-09-18T01:04:46.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.317678+0000", 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263479 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.318783+0000", 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011047190000000001 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:46.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319263+0000", 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000479576 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319305+0000", 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589000000000002e-05 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:43.319403+0000", 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8142999999999999e-05 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:24948 3.9 3:945f9464:::rbd_data.1103102c66ed.000000000000015d:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.279771162999999, 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89682947899999998, 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:46.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 24948 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448235+0000", 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448239+0000", 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1579999999999998e-06 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448250+0000", 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0061000000000001e-05 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448256+0000", 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6699999999999997e-06 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448310+0000", 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3888999999999999e-05 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448321+0000", 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1245e-05 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.342977+0000", 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89465616699999995 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343091+0000", 2024-09-18T01:04:46.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011382099999999999 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343168+0000", 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7015999999999996e-05 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.343923+0000", 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075445099999999995 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344936+0000", 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010128820000000001 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345011+0000", 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5519999999999995e-05 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345064+0000", 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2913e-05 2024-09-18T01:04:46.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:24951 3.9 3:90848b56:::rbd_data.1103102c66ed.00000000000001c1:head [delete] snapc 10=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.279595404999998, 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89740894000000004, 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 24951 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448410+0000", 2024-09-18T01:04:46.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448411+0000", 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6400000000000002e-07 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448415+0000", 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-06 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448422+0000", 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448427+0000", 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.313e-06 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448481+0000", 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4604000000000001e-05 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:27.448492+0000", 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0560000000000001e-05 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344058+0000", 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.89556604799999995 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344133+0000", 2024-09-18T01:04:46.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000006e-05 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.344202+0000", 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589000000000001e-05 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345247+0000", 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010448110000000001 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345693+0000", 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000446203 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345754+0000", 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0445999999999997e-05 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:28.345819+0000", 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5568999999999995e-05 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:26172 3.10 3:0a0d7897:::rbd_data.1103ca939bf9.000000000000007f:head [delete] snapc 9=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e115)", 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:36.366948+0000", 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.361057557000001, 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0286951040000001, 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:46.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 26172 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366948+0000", 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366948+0000", 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:46.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366950+0000", 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.716e-06 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366957+0000", 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8170000000000001e-06 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:46.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366970+0000", 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2814e-05 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.366981+0000", 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.151e-05 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.367048+0000", 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7156999999999994e-05 2024-09-18T01:04:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.367056+0000", 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2199999999999992e-06 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.368937+0000", 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001880187 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:36.368948+0000", 2024-09-18T01:04:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1223000000000001e-05 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.393596+0000", 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.024648378 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.393759+0000", 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163108 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:46.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.393910+0000", 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015027700000000001 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.394822+0000", 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091247099999999999 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.395467+0000", 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064444000000000003 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.395530+0000", 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3250000000000006e-05 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.395643+0000", 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011353600000000001 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:46.841 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:46.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T01:04:46.950 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:46.950 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:46.950 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-18T01:04:46.950 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:46.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T01:04:47.065 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-18T01:04:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-18T01:04:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-18T01:04:47.066 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:47.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T01:04:47.183 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:04:47.183 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-18T01:04:47.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-18T01:04:47.183 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-18T01:04:47.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.183 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22216 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.446465054, 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069476485000000004, 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22216 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.184 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734246+0000", 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734247+0000", 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734250+0000", 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0010000000000002e-06 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734256+0000", 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2230000000000004e-06 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734261+0000", 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7960000000000003e-06 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795021+0000", 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060760727 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795111+0000", 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9276000000000004e-05 2024-09-18T01:04:47.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795346+0000", 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023490100000000001 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.800389+0000", 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050430420000000002 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803654+0000", 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032650700000000001 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803683+0000", 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9286999999999999e-05 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803723+0000", 2024-09-18T01:04:47.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9715000000000002e-05 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22217 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.44644509600001, 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069528020999999995, 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22217 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734266+0000", 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734267+0000", 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734269+0000", 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734275+0000", 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734280+0000", 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890000000000002e-06 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795913+0000", 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061633895000000001 2024-09-18T01:04:47.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.795987+0000", 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3223e-05 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796338+0000", 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000351305 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803646+0000", 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007307872 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803730+0000", 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3677999999999994e-05 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803755+0000", 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5695999999999999e-05 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803794+0000", 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909000000000003e-05 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22218 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.446426155, 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069658842999999998, 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22218 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734285+0000", 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-18T01:04:47.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734288+0000", 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4370000000000001e-06 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734295+0000", 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3669999999999999e-06 2024-09-18T01:04:47.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734300+0000", 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796803+0000", 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062502809000000006 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.796853+0000", 2024-09-18T01:04:47.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9849000000000003e-05 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.797047+0000", 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019403700000000001 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803801+0000", 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067535970000000001 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803848+0000", 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7521000000000001e-05 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803904+0000", 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6146000000000001e-05 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803944+0000", 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9712999999999997e-05 2024-09-18T01:04:47.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22219 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.446404646, 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069771103000000001, 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22219 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:47.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734307+0000", 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734310+0000", 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734315+0000", 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6690000000000003e-06 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734323+0000", 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8019999999999992e-06 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798031+0000", 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063707689999999997 2024-09-18T01:04:47.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798079+0000", 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7979000000000003e-05 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798280+0000", 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020170499999999999 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803804+0000", 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055236349999999998 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804001+0000", 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019670000000000001 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804045+0000", 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4759999999999998e-05 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804078+0000", 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2242000000000001e-05 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22220 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:47.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.44637891799999, 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069876487000000001, 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22220 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734332+0000", 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734333+0000", 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734336+0000", 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8439999999999998e-06 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734342+0000", 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7560000000000002e-06 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.734348+0000", 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798921+0000", 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064573635000000004 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.798997+0000", 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5766000000000006e-05 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.799350+0000", 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035285999999999998 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.803807+0000", 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044571869999999996 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804123+0000", 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031632999999999997 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804174+0000", 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0467999999999999e-05 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804209+0000", 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5006999999999999e-05 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22228 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437624345, 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069779233999999996, 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22228 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743087+0000", 2024-09-18T01:04:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743090+0000", 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743096+0000", 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7579999999999998e-06 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743100+0000", 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5469999999999998e-06 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804657+0000", 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061556531999999997 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.804731+0000", 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3829000000000001e-05 2024-09-18T01:04:47.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805050+0000", 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031931400000000001 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811702+0000", 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066521849999999997 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812768+0000", 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010662029999999999 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812844+0000", 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5329999999999999e-05 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812866+0000", 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2555e-05 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22229 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437605215, 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.069890068999999999, 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22229 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743106+0000", 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743107+0000", 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743109+0000", 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743114+0000", 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-09-18T01:04:47.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743119+0000", 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805771+0000", 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062651472999999999 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.805862+0000", 2024-09-18T01:04:47.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1353000000000004e-05 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806155+0000", 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000292358 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811709+0000", 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055548100000000003 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812920+0000", 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001210842 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812956+0000", 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5361000000000002e-05 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812996+0000", 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0555e-05 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22230 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437586369, 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070047643000000007, 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22230 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743125+0000", 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743128+0000", 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743133+0000", 2024-09-18T01:04:47.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743138+0000", 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3290000000000004e-06 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806693+0000", 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063555005999999997 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806740+0000", 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6941999999999999e-05 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.806943+0000", 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020344499999999999 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811715+0000", 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047723710000000001 2024-09-18T01:04:47.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813047+0000", 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013317540000000001 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813132+0000", 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5168999999999997e-05 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813173+0000", 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0218e-05 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22231 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437567491, 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070186916000000002, 2024-09-18T01:04:47.235 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22231 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743144+0000", 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743147+0000", 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2909999999999999e-06 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743153+0000", 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6670000000000001e-06 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743158+0000", 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1120000000000004e-06 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807360+0000", 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.064201967999999998 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807422+0000", 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1826000000000001e-05 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.807895+0000", 2024-09-18T01:04:47.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047249999999999999 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812878+0000", 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049831830000000004 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813218+0000", 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034061699999999998 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813290+0000", 2024-09-18T01:04:47.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1567000000000001e-05 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813331+0000", 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0760999999999998e-05 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22232 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437545626, 2024-09-18T01:04:47.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070333498999999994, 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22232 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743166+0000", 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743169+0000", 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7839999999999999e-06 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743176+0000", 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743181+0000", 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6740000000000002e-06 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808365+0000", 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065183465999999995 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808409+0000", 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410999999999999e-05 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.808594+0000", 2024-09-18T01:04:47.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018510299999999999 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812881+0000", 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042869850000000001 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813411+0000", 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052974499999999998 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813461+0000", 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0099000000000003e-05 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813499+0000", 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8009999999999997e-05 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22233 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437521991, 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070465397999999999, 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22233 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743189+0000", 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743190+0000", 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-18T01:04:47.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743192+0000", 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5890000000000001e-06 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743201+0000", 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4689999999999997e-06 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743206+0000", 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2959999999999998e-06 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809109+0000", 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.065903223999999996 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809154+0000", 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4551000000000001e-05 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809342+0000", 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018778499999999999 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812884+0000", 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035426699999999999 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813563+0000", 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067874000000000005 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813618+0000", 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5043000000000001e-05 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813655+0000", 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6599e-05 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22234 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.43748834100001, 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070588851999999994, 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22234 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743223+0000", 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743224+0000", 2024-09-18T01:04:47.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-07 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743227+0000", 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.422e-06 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743236+0000", 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6780000000000006e-06 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743242+0000", 2024-09-18T01:04:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1199999999999999e-06 2024-09-18T01:04:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809858+0000", 2024-09-18T01:04:47.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.066616647000000001 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.809936+0000", 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7824999999999994e-05 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810140+0000", 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203749 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813348+0000", 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032080630000000001 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813721+0000", 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037242200000000002 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813767+0000", 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6900999999999997e-05 2024-09-18T01:04:47.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813812+0000", 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4292000000000002e-05 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22235 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:47.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.43746048600001, 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074915047999999998, 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22235 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:47.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743251+0000", 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743254+0000", 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7800000000000001e-06 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743261+0000", 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743265+0000", 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2320000000000003e-06 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810720+0000", 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.067455423 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.810794+0000", 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4204999999999999e-05 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811114+0000", 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031975100000000001 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818066+0000", 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069517520000000003 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818112+0000", 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6343e-05 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818143+0000", 2024-09-18T01:04:47.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0496000000000001e-05 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818166+0000", 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3059999999999999e-05 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22236 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437440663, 2024-09-18T01:04:47.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.074987463000000004, 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22236 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743271+0000", 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743274+0000", 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5270000000000001e-06 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743279+0000", 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743284+0000", 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1150000000000002e-06 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811759+0000", 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.068475076999999995 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.811855+0000", 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5165000000000001e-05 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.812233+0000", 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037825199999999998 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818073+0000", 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058399280000000003 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818206+0000", 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328999999999999 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818233+0000", 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7304000000000002e-05 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818258+0000", 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000001e-05 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22237 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437420229, 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075048580000000004, 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22237 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743291+0000", 2024-09-18T01:04:47.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743294+0000", 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743301+0000", 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3509999999999998e-06 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743306+0000", 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3229999999999997e-06 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813855+0000", 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070549385000000006 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.813918+0000", 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2943999999999999e-05 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814121+0000", 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020317600000000001 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818266+0000", 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041446060000000003 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818295+0000", 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8693000000000001e-05 2024-09-18T01:04:47.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818321+0000", 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6016999999999998e-05 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818340+0000", 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8929000000000001e-05 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22238 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437398149, 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.075127279000000005, 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22238 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743313+0000", 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743314+0000", 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-18T01:04:47.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743316+0000", 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.745e-06 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743323+0000", 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743328+0000", 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.091e-06 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814643+0000", 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071314218999999998 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814713+0000", 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0098999999999994e-05 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.814901+0000", 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000188488 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818269+0000", 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003368149 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818378+0000", 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108639 2024-09-18T01:04:47.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818405+0000", 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6514999999999999e-05 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.818440+0000", 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5803000000000003e-05 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22239 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437375827, 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.077753757000000007, 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22239 2024-09-18T01:04:47.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743335+0000", 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743336+0000", 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743339+0000", 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6919999999999998e-06 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743345+0000", 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743350+0000", 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0000000000000004e-06 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815337+0000", 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071987158999999995 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815426+0000", 2024-09-18T01:04:47.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8546000000000002e-05 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.815632+0000", 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000206066 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.820933+0000", 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053010100000000001 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821048+0000", 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114747 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821071+0000", 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3118000000000001e-05 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821089+0000", 2024-09-18T01:04:47.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8394e-05 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22240 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.43735343100001, 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078585004999999999, 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22240 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:47.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743358+0000", 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743361+0000", 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8830000000000002e-06 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743368+0000", 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2960000000000004e-06 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743373+0000", 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816262+0000", 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072889150999999999 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816341+0000", 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.928e-05 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.816660+0000", 2024-09-18T01:04:47.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031894699999999998 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821846+0000", 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005186141 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821891+0000", 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5222000000000002e-05 2024-09-18T01:04:47.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821921+0000", 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9946000000000001e-05 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821943+0000", 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1577999999999999e-05 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4642.0:22241 3.7 3:e0574372:::rbd_object_map.1103ca939bf9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.437333012, 2024-09-18T01:04:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.078722919000000002, 2024-09-18T01:04:47.298 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4642", 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22241 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743378+0000", 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743379+0000", 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743381+0000", 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4980000000000001e-06 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743388+0000", 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.743393+0000", 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817261+0000", 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073867622999999993 2024-09-18T01:04:47.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817346+0000", 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5405e-05 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.817584+0000", 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237669 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-18T01:04:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821856+0000", 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042721469999999996 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.821992+0000", 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013591499999999999 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822055+0000", 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3410000000000004e-05 2024-09-18T01:04:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:03:01.822101+0000", 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6031e-05 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4355.0:26193 3.7 3.4ec2ea07 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-18T01:04:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-18T01:04:37.401191+0000", 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7795205890000005, 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.88800681800000003, 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4355", 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 26193 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-18T01:04:47.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401191+0000", 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401191+0000", 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401192+0000", 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1319999999999999e-06 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401198+0000", 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8660000000000002e-06 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401209+0000", 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1341e-05 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401217+0000", 2024-09-18T01:04:47.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5689999999999999e-06 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401296+0000", 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9853999999999998e-05 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:37.401308+0000", 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2048999999999999e-05 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:38.289186+0000", 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.88787759700000002 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-18T01:04:38.289197+0000", 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1409999999999999e-05 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-18T01:04:47.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-18T01:04:47.311 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:04:47.312 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T01:04:47.423 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:47.423 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:47.424 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:47.431 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:04:47.432 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:04:47.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T01:04:47.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:04:47.536 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:47.537 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:47.537 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:47.537 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:47.537 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:47.545 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23894 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894835296] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.604461836, 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037953973000000002, 2024-09-18T01:04:47.650 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23894 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043617+0000", 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3700000000000001e-07 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.652 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043619+0000", 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7969999999999999e-06 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043622+0000", 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3610000000000001e-06 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043625+0000", 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6709999999999999e-06 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.653 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081473+0000", 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037848286000000002 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081507+0000", 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3785999999999998e-05 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081571+0000", 2024-09-18T01:04:47.654 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3734999999999999e-05 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23895 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894846976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.604450685, 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038153152000000003, 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.655 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23895 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.656 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043628+0000", 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.4600000000000001e-07 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043630+0000", 2024-09-18T01:04:47.657 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5910000000000001e-06 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043633+0000", 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5269999999999999e-06 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.658 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043636+0000", 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5840000000000002e-06 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081601+0000", 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037965437999999997 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.659 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081664+0000", 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2728e-05 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081781+0000", 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117038 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23896 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894858768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.604438855, 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038250711999999999, 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23896 2024-09-18T01:04:47.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043640+0000", 2024-09-18T01:04:47.662 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043641+0000", 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.455e-06 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043644+0000", 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9170000000000002e-06 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.663 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043653+0000", 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5839999999999995e-06 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081793+0000", 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038140476999999999 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081826+0000", 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2835e-05 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.664 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081890+0000", 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4220000000000005e-05 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23897 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894871056] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.604422172, 2024-09-18T01:04:47.665 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038447454999999998, 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23897 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:47.666 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043656+0000", 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2399999999999999e-07 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.667 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043658+0000", 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9400000000000001e-06 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043662+0000", 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2229999999999999e-06 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043665+0000", 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0350000000000002e-06 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.668 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081926+0000", 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038261306000000002 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.081988+0000", 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2107999999999996e-05 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.669 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082104+0000", 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011561900000000001 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23898 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894883328] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.604410935, 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038543726, 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.670 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23898 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.671 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.043667+0000", 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044910+0000", 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001242235 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044913+0000", 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3979999999999999e-06 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.672 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044924+0000", 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1072e-05 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044930+0000", 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6540000000000004e-06 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082117+0000", 2024-09-18T01:04:47.673 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037186874000000002 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082148+0000", 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1367e-05 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082211+0000", 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3126000000000005e-05 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.674 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23903 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894924544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.60308883, 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038079969999999998, 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.675 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23903 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.676 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044990+0000", 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1e-07 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044991+0000", 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5939999999999999e-06 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044994+0000", 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.993e-06 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.677 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.044998+0000", 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2830000000000002e-06 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082885+0000", 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037887151000000001 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.082954+0000", 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8728999999999999e-05 2024-09-18T01:04:47.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083070+0000", 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011601 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23904 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894952384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.60307611499999, 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038193997, 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.679 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23904 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.680 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045002+0000", 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045003+0000", 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7799999999999997e-07 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045005+0000", 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1639999999999999e-06 2024-09-18T01:04:47.681 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045010+0000", 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5380000000000002e-06 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045015+0000", 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5290000000000002e-06 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.682 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083097+0000", 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038081994000000001 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083131+0000", 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4181999999999999e-05 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083196+0000", 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5312000000000001e-05 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.683 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23905 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894964816] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.60303113099999, 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038360660999999997, 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23905 2024-09-18T01:04:47.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045047+0000", 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.685 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045048+0000", 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1399999999999997e-07 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045052+0000", 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5340000000000001e-06 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045060+0000", 2024-09-18T01:04:47.686 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8259999999999993e-06 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045066+0000", 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1639999999999998e-06 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083226+0000", 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038159684999999999 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.687 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083289+0000", 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3393999999999996e-05 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083408+0000", 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118544 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23906 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894976368] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.688 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.603003507, 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038443024999999999, 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23906 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.689 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.045075+0000", 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046924+0000", 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001848798 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046928+0000", 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.941e-06 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046935+0000", 2024-09-18T01:04:47.691 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9029999999999998e-06 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.046939+0000", 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.6090000000000003e-06 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083419+0000", 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036480209 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083454+0000", 2024-09-18T01:04:47.692 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4956999999999998e-05 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.083518+0000", 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3608000000000005e-05 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23925 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894007712] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.693 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.599456128, 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038058209000000003, 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.694 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23925 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048622+0000", 2024-09-18T01:04:47.695 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048924+0000", 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030148099999999998 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048927+0000", 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2009999999999999e-06 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.696 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048932+0000", 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4099999999999999e-06 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048938+0000", 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3569999999999997e-06 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.697 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086500+0000", 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037562110000000003 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086563+0000", 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3304000000000002e-05 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086681+0000", 2024-09-18T01:04:47.698 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000117346 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23927 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894690864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.59909602099999, 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038025200000000002, 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.699 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23927 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048982+0000", 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048983+0000", 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7299999999999996e-07 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048986+0000", 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.8370000000000001e-06 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048992+0000", 2024-09-18T01:04:47.701 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7819999999999999e-06 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.048998+0000", 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7129999999999999e-06 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086820+0000", 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037821894000000002 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.702 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.086885+0000", 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4993000000000006e-05 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087008+0000", 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00012240800000000001 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23928 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894701344] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.599069908, 2024-09-18T01:04:47.703 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038108283999999999, 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23928 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:47.704 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.049009+0000", 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050927+0000", 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019179970000000001 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.705 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.050930+0000", 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8179999999999997e-06 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051029+0000", 2024-09-18T01:04:47.706 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.8593999999999998e-05 2024-09-18T01:04:47.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.707 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051034+0000", 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3249999999999998e-06 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087020+0000", 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.035985376999999999 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087053+0000", 2024-09-18T01:04:47.708 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3263000000000003e-05 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.087117+0000", 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3910000000000003e-05 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23941 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894174704] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.59689052, 2024-09-18T01:04:47.709 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038106243999999997, 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23941 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051188+0000", 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051190+0000", 2024-09-18T01:04:47.711 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.514e-06 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051193+0000", 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9639999999999999e-06 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051195+0000", 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.3410000000000001e-06 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089112+0000", 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037917215999999997 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089177+0000", 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4752000000000006e-05 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089294+0000", 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011725100000000001 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23942 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894186224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.714 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.59688072500001, 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038203687, 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23942 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.715 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051198+0000", 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0100000000000001e-07 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051199+0000", 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5650000000000001e-06 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051202+0000", 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.853e-06 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051206+0000", 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3119999999999998e-06 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089306+0000", 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038100281999999999 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089337+0000", 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.0929999999999997e-05 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089401+0000", 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4543999999999997e-05 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23943 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894198096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.596869946, 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038399142999999997, 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.719 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23943 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.720 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051208+0000", 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051209+0000", 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0599999999999999e-07 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051214+0000", 2024-09-18T01:04:47.721 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5280000000000003e-06 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051220+0000", 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6470000000000003e-06 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051224+0000", 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8340000000000001e-06 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089431+0000", 2024-09-18T01:04:47.722 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038207682 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089494+0000", 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3116999999999999e-05 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089608+0000", 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00011312899999999999 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.723 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23944 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140535894208576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.59685014999999, 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038507929000000003, 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.724 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23944 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.725 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.051228+0000", 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052925+0000", 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0016965000000000001 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052928+0000", 2024-09-18T01:04:47.726 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7470000000000001e-06 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052934+0000", 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2620000000000004e-06 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.052937+0000", 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6179999999999999e-06 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.727 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089635+0000", 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.036697186999999999 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089670+0000", 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5302999999999997e-05 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.089736+0000", 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6311999999999998e-05 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.728 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23961 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [notify cookie 140536230226352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.59503979199999, 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038021372999999997, 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.729 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23961 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053039+0000", 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2700000000000001e-07 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053040+0000", 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.564e-06 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053043+0000", 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.9689999999999999e-06 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053046+0000", 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.6280000000000001e-06 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090949+0000", 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.037902874000000003 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.732 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.090985+0000", 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6003000000000001e-05 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091060+0000", 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5108e-05 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.733 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4642.0:23962 3.f 3:f64b92a0:::rbd_header.1103ca939bf9:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 105.595029244, 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038456549, 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4642", 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1304046458", 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 23962 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.734 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053049+0000", 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.17e-07 2024-09-18T01:04:47.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053054+0000", 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.972e-06 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053058+0000", 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1329999999999999e-06 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.736 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.053060+0000", 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.65e-06 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091092+0000", 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.038031941999999999 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091176+0000", 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3788999999999993e-05 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.737 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:02.091506+0000", 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00032984600000000001 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.5042.0:2934 3.16 3:6ec7e9e7:::rbd_data.11034a3194e.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 70.123102368000005, 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99770834600000002, 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.738 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.5042", 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/3621248646", 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 2934 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524976+0000", 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524977+0000", 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.142e-06 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524981+0000", 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9809999999999997e-06 2024-09-18T01:04:47.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524992+0000", 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1028e-05 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.524998+0000", 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4580000000000001e-06 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.741 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525056+0000", 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.8121999999999997e-05 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:37.525067+0000", 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1228e-05 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522447+0000", 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.99738034900000005 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.742 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522547+0000", 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9884999999999997e-05 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:38.522684+0000", 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013715299999999999 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:26149 3.1b 3:d8d208d4:::rbd_data.1103ca939bf9.0000000000000020:head [delete] snapc 9=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e115)", 2024-09-18T01:04:47.743 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:04:36.366973+0000", 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 11.281105736000001, 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0278071440000001, 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 26149 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366973+0000", 2024-09-18T01:04:47.744 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366973+0000", 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366975+0000", 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2189999999999998e-06 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:47.745 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.366984+0000", 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7690000000000002e-06 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367005+0000", 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1747e-05 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367019+0000", 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3617e-05 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367101+0000", 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2379000000000005e-05 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.367110+0000", 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.884e-06 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.747 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.370160+0000", 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0030499640000000001 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:36.370168+0000", 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2470000000000002e-06 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.392709+0000", 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0225411740000001 2024-09-18T01:04:47.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.392834+0000", 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000125034 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0,4", 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.392949+0000", 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000114799 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.393531+0000", 2024-09-18T01:04:47.749 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00058285699999999999 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.393963+0000", 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00043142799999999998 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.394674+0000", 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00071089199999999997 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.394717+0000", 2024-09-18T01:04:47.750 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3754999999999997e-05 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:04:37.394780+0000", 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2378999999999993e-05 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:47.751 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:47.752 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T01:04:47.855 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:47.855 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:47.855 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:47.855 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:47.864 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T01:04:47.967 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:47.967 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:47.967 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:47.976 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T01:04:48.079 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:48.079 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:48.079 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7460 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.781787145, 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023494623999999999, 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.080 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7460 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295967+0000", 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.081 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295968+0000", 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9500000000000002e-07 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295970+0000", 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.26e-06 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295978+0000", 2024-09-18T01:04:48.082 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.7270000000000007e-06 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295984+0000", 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1750000000000002e-06 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319454+0000", 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023469365999999998 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.083 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319462+0000", 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5010000000000004e-06 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7461 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.781763028, 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023515778000000001, 2024-09-18T01:04:48.084 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7461 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:48.085 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295992+0000", 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0900000000000002e-07 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.086 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.295994+0000", 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9360000000000002e-06 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296001+0000", 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7730000000000002e-06 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296006+0000", 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2630000000000003e-06 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319495+0000", 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023488990000000001 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319507+0000", 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2306999999999999e-05 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.088 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7474 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.781683788, 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461554999999999, 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7474 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.089 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296071+0000", 2024-09-18T01:04:48.090 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.0200000000000002e-07 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296074+0000", 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2809999999999998e-06 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296081+0000", 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6079999999999998e-06 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.091 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296087+0000", 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5929999999999996e-06 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319524+0000", 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023436741000000001 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319532+0000", 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.7299999999999994e-06 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.092 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7475 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.781647569, 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023465426000000001, 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.093 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7475 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.094 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296107+0000", 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2099999999999997e-07 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296114+0000", 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6619999999999999e-06 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296122+0000", 2024-09-18T01:04:48.095 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2179999999999999e-06 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296128+0000", 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6030000000000004e-06 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.096 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319562+0000", 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023433656000000001 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319572+0000", 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0766e-05 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7478 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.097 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.781618956, 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023461356999999999, 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7478 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.098 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296136+0000", 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.01e-07 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296138+0000", 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2560000000000001e-06 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296145+0000", 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4330000000000001e-06 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296150+0000", 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1499999999999998e-06 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319589+0000", 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023439056 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319597+0000", 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0609999999999996e-06 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.101 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7480 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78159809100001, 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02347925, 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.102 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7480 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296156+0000", 2024-09-18T01:04:48.103 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296157+0000", 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0200000000000003e-07 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296159+0000", 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.877e-06 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.104 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296165+0000", 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3770000000000001e-06 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296170+0000", 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1930000000000001e-06 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319625+0000", 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023454914 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319636+0000", 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0487e-05 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7490 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78098037399999, 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.02288662, 2024-09-18T01:04:48.106 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7490 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296774+0000", 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296775+0000", 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4099999999999998e-07 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296778+0000", 2024-09-18T01:04:48.108 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2650000000000001e-06 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296786+0000", 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3550000000000005e-06 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296793+0000", 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7900000000000002e-06 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.109 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319653+0000", 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859708999999999 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319661+0000", 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8599999999999993e-06 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.110 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7493 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78095110700001, 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022885419000000001, 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7493 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.111 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296803+0000", 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296804+0000", 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:48.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296806+0000", 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1519999999999999e-06 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296815+0000", 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5529999999999993e-06 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.113 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296820+0000", 2024-09-18T01:04:48.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.7010000000000001e-06 2024-09-18T01:04:48.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.114 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.114 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.114 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319679+0000", 2024-09-18T01:04:48.114 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022858753999999998 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319689+0000", 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.781e-06 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.116 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7501 3.3 3.5c46bd03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.780884816, 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022878483000000002, 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7501 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.117 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296870+0000", 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8599999999999999e-07 2024-09-18T01:04:48.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296872+0000", 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.871e-06 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296892+0000", 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9775999999999998e-05 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296898+0000", 2024-09-18T01:04:48.119 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2670000000000003e-06 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319738+0000", 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022840124 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319748+0000", 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0059e-05 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.120 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7512 3.3 3.fbf58703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.780764049, 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.023630136999999999, 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.121 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7512 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296990+0000", 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.122 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.296991+0000", 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0999999999999999e-07 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297003+0000", 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297010+0000", 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6149999999999999e-06 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.123 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297016+0000", 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6080000000000003e-06 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.319799+0000", 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022783437 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320621+0000", 2024-09-18T01:04:48.124 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00082117799999999999 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7601 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78054560299999, 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869878, 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.125 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7601 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:48.126 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297209+0000", 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297211+0000", 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0030000000000001e-06 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.127 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297219+0000", 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4880000000000001e-06 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297224+0000", 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.78e-06 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.128 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320072+0000", 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022848383999999999 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320079+0000", 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8519999999999998e-06 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7612 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.780524242, 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022873074, 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7612 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.130 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297230+0000", 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297231+0000", 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9099999999999999e-07 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.131 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297233+0000", 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1009999999999999e-06 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297239+0000", 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6780000000000003e-06 2024-09-18T01:04:48.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297245+0000", 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3179999999999997e-06 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320097+0000", 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022851776000000001 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.133 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320103+0000", 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.81e-06 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7661 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.780370303, 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022869843000000001, 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.134 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7661 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.135 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297384+0000", 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297385+0000", 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.89e-07 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297387+0000", 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0999999999999998e-06 2024-09-18T01:04:48.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297393+0000", 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.3579999999999999e-06 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297398+0000", 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8110000000000002e-06 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.137 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320247+0000", 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022849273999999999 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320254+0000", 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9110000000000004e-06 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7675 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:48.138 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78035058499999, 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022875395, 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7675 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.139 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297404+0000", 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297405+0000", 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9999999999999997e-07 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.140 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297406+0000", 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9e-06 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297413+0000", 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4919999999999998e-06 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297418+0000", 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1209999999999998e-06 2024-09-18T01:04:48.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320273+0000", 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022854436999999998 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320279+0000", 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8449999999999997e-06 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7681 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.780330275, 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879914000000001, 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7681 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.143 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297424+0000", 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.144 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297425+0000", 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8200000000000001e-07 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297427+0000", 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0839999999999999e-06 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297433+0000", 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1890000000000004e-06 2024-09-18T01:04:48.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297438+0000", 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0509999999999996e-06 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320297+0000", 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022859344 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320304+0000", 2024-09-18T01:04:48.146 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8639999999999998e-06 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7689 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78031038899999, 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022886786999999999, 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.147 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7689 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.148 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297444+0000", 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297445+0000", 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7800000000000002e-07 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297447+0000", 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.962e-06 2024-09-18T01:04:48.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297453+0000", 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.455e-06 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297458+0000", 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.011e-06 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320324+0000", 2024-09-18T01:04:48.150 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022865856 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320331+0000", 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0249999999999997e-06 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7704 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:48.151 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78029042999999, 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022893301000000001, 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7704 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.152 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297464+0000", 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297465+0000", 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9400000000000001e-07 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.153 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297467+0000", 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.266e-06 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297473+0000", 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.4980000000000003e-06 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297478+0000", 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8409999999999999e-06 2024-09-18T01:04:48.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320350+0000", 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022872092 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320357+0000", 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2099999999999996e-06 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7705 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78027043199999, 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022899778999999999, 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7705 2024-09-18T01:04:48.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297484+0000", 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.157 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297485+0000", 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0600000000000001e-07 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297486+0000", 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9460000000000001e-06 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297493+0000", 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2979999999999997e-06 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.158 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297498+0000", 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0989999999999998e-06 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320377+0000", 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022879040999999999 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320384+0000", 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9890000000000003e-06 2024-09-18T01:04:48.159 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7718 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.78025076500001, 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022905915999999998, 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.160 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7718 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297504+0000", 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.7099999999999997e-07 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297506+0000", 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8619999999999999e-06 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297513+0000", 2024-09-18T01:04:48.162 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5400000000000001e-06 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297518+0000", 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.9679999999999997e-06 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320402+0000", 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022884976000000001 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.163 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320410+0000", 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.199e-06 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4726.0:7726 3.3 3.177f5703 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 101.780230863, 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022909993999999999, 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.164 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "reached pg", 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4726", 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/562645976", 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7726 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.165 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297524+0000", 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9299999999999999e-07 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297531+0000", 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.5069999999999997e-06 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.166 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297538+0000", 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.0029999999999997e-06 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.297543+0000", 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1479999999999997e-06 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320427+0000", 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.022883769000000002 2024-09-18T01:04:48.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:03:06.320434+0000", 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1740000000000003e-06 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.168 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:48.170 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T01:04:48.273 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:48.273 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:48.273 INFO:teuthology.orchestra.run.smithi098.stdout: "num_ops": 0 2024-09-18T01:04:48.273 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:48.282 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T01:04:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:04:48.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:04:48.388 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [], 2024-09-18T01:04:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "complaint_time": 30, 2024-09-18T01:04:48.388 INFO:teuthology.orchestra.run.smithi098.stdout: "num_blocked_ops": 0 2024-09-18T01:04:48.388 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:48.396 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: "size": 20, 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 600, 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: "ops": [ 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 136.12505559300001, 2024-09-18T01:04:48.501 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0084069060000000004, 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373946+0000", 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373950+0000", 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.2300000000000002e-06 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373957+0000", 2024-09-18T01:04:48.503 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.153e-06 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373973+0000", 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.5824e-05 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.373994+0000", 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1016999999999999e-05 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.504 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374090+0000", 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5865000000000005e-05 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for peered", 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.374100+0000", 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0729e-05 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377516+0000", 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0034157250000000001 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377672+0000", 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00015530900000000001 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 2", 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.377849+0000", 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017697899999999999 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.506 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382248+0000", 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043997400000000001 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382285+0000", 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6340000000000001e-05 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382321+0000", 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6433999999999997e-05 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.507 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:32.382353+0000", 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1560999999999998e-05 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 135.12704428800001, 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050996230000000002, 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.508 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 5 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.509 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371957+0000", 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371959+0000", 2024-09-18T01:04:48.510 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.1880000000000001e-06 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371971+0000", 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1419e-05 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371989+0000", 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.8218000000000001e-05 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.511 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.371999+0000", 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.005e-05 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372085+0000", 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5829999999999996e-05 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.372096+0000", 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1399999999999999e-05 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375218+0000", 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0031217829999999999 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375328+0000", 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000109826 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.375437+0000", 2024-09-18T01:04:48.513 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000108902 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376267+0000", 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00083011100000000002 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.376967+0000", 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070003700000000003 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.514 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377005+0000", 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8448000000000002e-05 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:33.377057+0000", 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1411e-05 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:48.515 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 134.11936733600001, 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051063940000000002, 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4208", 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.31:0/3183163727", 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 7 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.516 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379634+0000", 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379636+0000", 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.683e-06 2024-09-18T01:04:48.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379645+0000", 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.4849999999999998e-06 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379663+0000", 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.7799999999999999e-05 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379673+0000", 2024-09-18T01:04:48.518 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.9210000000000004e-06 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379736+0000", 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2737999999999995e-05 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.379741+0000", 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0250000000000002e-06 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.519 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381474+0000", 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0017331320000000001 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381585+0000", 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111498 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 1", 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.381763+0000", 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017738199999999999 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.520 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.383098+0000", 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013351280000000001 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384649+0000", 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001550884 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384690+0000", 2024-09-18T01:04:48.521 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1029000000000003e-05 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:34.384740+0000", 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0689000000000002e-05 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4355.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 125.02150938600001, 2024-09-18T01:04:48.522 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0074814149999999999, 2024-09-18T01:04:48.523 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.523 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.523 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.523 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.523 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4355", 2024-09-18T01:04:48.523 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/220235413", 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 10 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477492+0000", 2024-09-18T01:04:48.524 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477494+0000", 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6789999999999999e-06 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477499+0000", 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1050000000000003e-06 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.525 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477508+0000", 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.1290000000000002e-06 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477516+0000", 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.157e-06 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477565+0000", 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8906000000000002e-05 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.526 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op must wait for map", 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.477569+0000", 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.946e-06 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479206+0000", 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001636848 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.527 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479381+0000", 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00017543800000000001 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.479634+0000", 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00025298200000000001 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.482421+0000", 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.002786972 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484873+0000", 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024518080000000002 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484929+0000", 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.6548000000000001e-05 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:43.484973+0000", 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3896999999999997e-05 2024-09-18T01:04:48.529 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:592 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.165428374, 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0019985329999999998, 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.530 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 592 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:48.531 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333573+0000", 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333575+0000", 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.866e-06 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.532 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333661+0000", 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.6007999999999994e-05 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333673+0000", 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2092999999999999e-05 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333679+0000", 2024-09-18T01:04:48.533 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.874e-06 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333733+0000", 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4534000000000003e-05 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333872+0000", 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00013843699999999999 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:48.534 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334040+0000", 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000168679 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335214+0000", 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001173428 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335499+0000", 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00028555900000000001 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.535 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335533+0000", 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.3260999999999999e-05 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335571+0000", 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8794000000000002e-05 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:594 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 148992~65024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.536 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.165059078, 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003996922, 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 594 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333942+0000", 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.333943+0000", 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2999999999999999e-07 2024-09-18T01:04:48.538 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334035+0000", 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2336999999999994e-05 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334047+0000", 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1789e-05 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334055+0000", 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9680000000000006e-06 2024-09-18T01:04:48.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334463+0000", 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00040830599999999999 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334547+0000", 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3720000000000005e-05 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.540 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.334716+0000", 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000169209 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335424+0000", 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00070773500000000003 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337863+0000", 2024-09-18T01:04:48.541 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0024393700000000002 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337899+0000", 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.5889000000000001e-05 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337939+0000", 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9768999999999998e-05 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.542 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:595 3.10 3:0c263d2a:::rbd_data.1103b05c2cf1.0000000000000124:head [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.16332023299999, 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021504179999999999, 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 595 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.543 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335681+0000", 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335682+0000", 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.2399999999999997e-07 2024-09-18T01:04:48.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335686+0000", 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4360000000000004e-06 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335694+0000", 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.2670000000000006e-06 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.545 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335700+0000", 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.2399999999999998e-06 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335747+0000", 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7160000000000002e-05 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335831+0000", 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.4142999999999997e-05 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.546 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337831+0000", 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020004480000000002 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop_reply(client.4489.0:594 3.10 e26/20 ondisk, result = 0)", 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.16324834300001, 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0021953319999999999, 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "started", 2024-09-18T01:04:48.547 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.548 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335753+0000", 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5599999999999995e-07 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335757+0000", 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.2660000000000002e-06 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335765+0000", 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5180000000000004e-06 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.549 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.335771+0000", 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.02e-06 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337855+0000", 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00208413 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337860+0000", 2024-09-18T01:04:48.550 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1919999999999998e-06 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.337948+0000", 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.865e-05 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:706 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [set-alloc-hint object_size 524288 write_size 524288,write 199168~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.07538233299999, 2024-09-18T01:04:48.551 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001426673, 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 706 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423619+0000", 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423620+0000", 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.6400000000000001e-07 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.553 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423667+0000", 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7410000000000002e-05 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423678+0000", 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0434999999999999e-05 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423683+0000", 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0309999999999998e-06 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.554 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423722+0000", 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9079000000000003e-05 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423793+0000", 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.1241000000000005e-05 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.423934+0000", 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00014075700000000001 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.555 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424580+0000", 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00064666300000000001 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.424953+0000", 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00037232500000000001 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425006+0000", 2024-09-18T01:04:48.556 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3220999999999997e-05 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.425046+0000", 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9746999999999997e-05 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:715 3.13 e26/20 3:cdf51090:::rbd_data.1103b05c2cf1.0000000000000021:head v 26'6, mlcod=26'3)", 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.069191074, 2024-09-18T01:04:48.557 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027535820000000001, 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429810+0000", 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.558 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429811+0000", 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.6000000000000003e-07 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429818+0000", 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8310000000000003e-06 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429830+0000", 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.199e-05 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.559 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429845+0000", 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.4861000000000001e-05 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429925+0000", 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.0171000000000002e-05 2024-09-18T01:04:48.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.429945+0000", 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.0559000000000001e-05 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432524+0000", 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0025788209999999998 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.432564+0000", 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9688999999999999e-05 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:735 3.10 3:0cd1b818:::rbd_data.1103b05c2cf1.000000000000002a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.06766138899999, 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052305709999999998, 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.562 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 735 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.563 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431340+0000", 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431341+0000", 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9699999999999995e-07 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431352+0000", 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1454000000000001e-05 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.564 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431364+0000", 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2189e-05 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431372+0000", 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.8590000000000005e-06 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431414+0000", 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1681000000000003e-05 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.565 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431464+0000", 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.0246e-05 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431523+0000", 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9293000000000003e-05 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435210+0000", 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0036868109999999999 2024-09-18T01:04:48.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436474+0000", 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0012640150000000001 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436528+0000", 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4239e-05 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436570+0000", 2024-09-18T01:04:48.567 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1987000000000003e-05 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:738 3.10 3:0c4595d1:::rbd_data.1103b05c2cf1.0000000000000031:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.0672817, 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049451540000000002, 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.568 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 738 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.569 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431720+0000", 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.7999999999999995e-07 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431723+0000", 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1970000000000001e-06 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.570 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431733+0000", 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.2480000000000002e-06 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431738+0000", 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4920000000000001e-06 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431783+0000", 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.5302000000000001e-05 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431859+0000", 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.5823999999999998e-05 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.431970+0000", 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000111 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.435214+0000", 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.003244065 2024-09-18T01:04:48.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436606+0000", 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0013919830000000001 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436637+0000", 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1007999999999999e-05 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.436665+0000", 2024-09-18T01:04:48.573 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.7355000000000002e-05 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:765 3.1d 3:bbd0987c:::rbd_data.1103b05c2cf1.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 393216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:48.574 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.051783297, 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0059618409999999998, 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 765 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:48.575 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447218+0000", 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447219+0000", 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.124e-06 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.576 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447231+0000", 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1688999999999999e-05 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447244+0000", 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3251000000000001e-05 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447252+0000", 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.4730000000000003e-06 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447296+0000", 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4317000000000003e-05 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447401+0000", 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000104749 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 4", 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447606+0000", 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020501699999999999 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.578 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.450378+0000", 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027727590000000001 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453083+0000", 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0027047479999999999 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453137+0000", 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3439000000000001e-05 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.579 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453180+0000", 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3275000000000002e-05 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:771 3.16 3:6d9c453e:::rbd_data.1103b05c2cf1.0000000000000015:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.05157653800001, 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0058248869999999999, 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.580 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 771 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.581 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447425+0000", 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447426+0000", 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.0189999999999999e-06 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447431+0000", 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3199999999999999e-06 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447443+0000", 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1881e-05 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447450+0000", 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.3409999999999997e-06 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447519+0000", 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9008000000000001e-05 2024-09-18T01:04:48.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447652+0000", 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000132948 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447862+0000", 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00020968699999999999 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448824+0000", 2024-09-18T01:04:48.584 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00096213199999999998 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453155+0000", 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043314160000000003 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453209+0000", 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.3424000000000002e-05 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.585 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453250+0000", 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0710999999999997e-05 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:768 3.13 e26/20 3:cc27f995:::rbd_data.1103b05c2cf1.0000000000000012:head v 26'7, mlcod=26'5)", 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.051158781, 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0053259880000000002, 2024-09-18T01:04:48.586 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.587 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447842+0000", 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447843+0000", 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.1500000000000003e-07 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447849+0000", 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4559999999999999e-06 2024-09-18T01:04:48.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447858+0000", 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9670000000000007e-06 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447864+0000", 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.251e-06 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.589 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447903+0000", 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.9436e-05 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.447926+0000", 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.2541999999999999e-05 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453127+0000", 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0052005719999999997 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.590 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453168+0000", 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1949000000000001e-05 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:767 3.2 e26/20 3:4748b501:::rbd_data.1103b05c2cf1.0000000000000011:head v 26'53, mlcod=26'51)", 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.050949543, 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051407199999999997, 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.591 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448052+0000", 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9100000000000004e-07 2024-09-18T01:04:48.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448057+0000", 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.1840000000000001e-06 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448065+0000", 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.3359999999999995e-06 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.593 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448071+0000", 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.1160000000000004e-06 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448119+0000", 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8384000000000003e-05 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448136+0000", 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6592999999999999e-05 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.594 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453154+0000", 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0050183040000000003 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453192+0000", 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8211999999999999e-05 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:769 3.17 e26/20 3:ef7f8d5a:::rbd_data.1103b05c2cf1.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-18T01:04:48.595 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.05088560900001, 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0051020129999999999, 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.596 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448116+0000", 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.7599999999999997e-07 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448120+0000", 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.3250000000000001e-06 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448127+0000", 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.8549999999999996e-06 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448132+0000", 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.1699999999999996e-06 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448181+0000", 2024-09-18T01:04:48.598 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8529000000000002e-05 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448198+0000", 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.6540000000000001e-05 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453179+0000", 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0049818730000000004 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.599 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453218+0000", 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.8244999999999997e-05 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_repop(client.4489.0:772 3.17 e26/20 3:eb7af09f:::rbd_data.1103b05c2cf1.0000000000000016:head v 26'4, mlcod=24'1)", 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 121.050137404, 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0043814489999999999, 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.600 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.601 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448864+0000", 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.4799999999999998e-07 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448869+0000", 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.0929999999999996e-06 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448877+0000", 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.5799999999999992e-06 2024-09-18T01:04:48.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448888+0000", 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.1182000000000001e-05 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448933+0000", 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4379000000000003e-05 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.603 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.448952+0000", 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.9199e-05 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453219+0000", 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0042675229999999996 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.453245+0000", 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5945e-05 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1053 3.f 3:f0bafcd1:::rbd_data.1103b05c2cf1.00000000000001cc:head [set-alloc-hint object_size 524288 write_size 524288,write 307200~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 120.791198599, 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.001399256, 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1053 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.605 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707803+0000", 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707804+0000", 2024-09-18T01:04:48.606 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.539e-06 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707848+0000", 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.4092999999999999e-05 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707862+0000", 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.3220999999999999e-05 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.607 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707868+0000", 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.9850000000000003e-06 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.707923+0000", 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 5.5423000000000001e-05 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708018+0000", 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 9.5439999999999994e-05 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.608 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 3", 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708108+0000", 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 8.9933999999999996e-05 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708836+0000", 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00072802200000000004 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709145+0000", 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00030850000000000002 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709176+0000", 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.1266999999999997e-05 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709202+0000", 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 2.5831999999999999e-05 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "description": "osd_op(client.4489.0:1055 3.c 3:3441368c:::rbd_data.1103b05c2cf1.0000000000000115:head [set-alloc-hint object_size 524288 write_size 524288,write 393216~28672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-18T01:04:48.610 INFO:teuthology.orchestra.run.smithi098.stdout: "initiated_at": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "age": 120.790334981, 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0020350699999999999, 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "continuous": false, 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "type_data": { 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client_info": { 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client": "client.4489", 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "client_addr": "172.21.15.164:0/1291339329", 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "tid": 1055 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "events": [ 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "initiated", 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:48.611 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "header_read", 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708666+0000", 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "throttled", 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708667+0000", 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.116e-06 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "all_read", 2024-09-18T01:04:48.612 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708704+0000", 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.6607999999999999e-05 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "dispatched", 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708716+0000", 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 1.2007e-05 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "queued_for_pg", 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708723+0000", 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.9789999999999996e-06 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "reached_pg", 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708786+0000", 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 6.2663999999999995e-05 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "started", 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.708904+0000", 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.000118398 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "waiting for subops from 0", 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.709007+0000", 2024-09-18T01:04:48.614 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.00010320800000000001 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "sub_op_commit_rec", 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710539+0000", 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 0.0015315800000000001 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "op_commit", 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710619+0000", 2024-09-18T01:04:48.615 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 7.9739000000000003e-05 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "commit_sent", 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710667+0000", 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 4.8504000000000002e-05 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "event": "done", 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "time": "2024-09-18T01:02:47.710701+0000", 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: "duration": 3.4267000000000002e-05 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.616 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-18T01:04:48.617 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-18T01:04:48.617 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-18T01:04:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:04:49.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T01:04:49.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:04:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-18T01:04:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:04:50.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T01:04:50.458 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:04:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:04:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-18T01:04:51.462 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:04:51.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:04:51.796 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:51.797 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:04:51.808 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-18T01:04:51.446441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265729,"num_read_kb":1200588,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1360,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":281492,"kb_used_data":7508,"kb_used_omap":7,"kb_used_meta":273848,"kb_avail":471577708,"statfs":{"total":483183820800,"available":482895572992,"internally_reserved":0,"allocated":7688192,"data_stored":4302207,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":7973,"internal_metadata":280420571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-65294183,"num_objects":-45,"num_object_clones":-6,"num_object_copies":-90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-45,"num_whiteouts":-5,"num_read":577,"num_read_kb":501,"num_write":132,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017991"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10515,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292565+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:04:48.292565+0000","last_peered":"2024-09-18T01:04:48.292565+0000","last_clean":"2024-09-18T01:04:48.292565+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:04:48.292565+0000","last_undegraded":"2024-09-18T01:04:48.292565+0000","last_fullsized":"2024-09-18T01:04:48.292565+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1e","version":"114'1473","reported_seq":7711,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327848+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:04:48.327848+0000","last_peered":"2024-09-18T01:04:48.327848+0000","last_clean":"2024-09-18T01:04:48.327848+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:04:48.327848+0000","last_undegraded":"2024-09-18T01:04:48.327848+0000","last_fullsized":"2024-09-18T01:04:48.327848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1d","version":"124'455","reported_seq":7427,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342121+0000","last_change":"2024-09-18T01:04:46.426329+0000","last_active":"2024-09-18T01:04:48.342121+0000","last_peered":"2024-09-18T01:04:48.342121+0000","last_clean":"2024-09-18T01:04:48.342121+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:04:48.342121+0000","last_undegraded":"2024-09-18T01:04:48.342121+0000","last_fullsized":"2024-09-18T01:04:48.342121+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:42.667965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500068,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1c","version":"125'869","reported_seq":12779,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342120+0000","last_change":"2024-09-18T01:04:46.427109+0000","last_active":"2024-09-18T01:04:48.342120+0000","last_peered":"2024-09-18T01:04:48.342120+0000","last_clean":"2024-09-18T01:04:48.342120+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:04:48.342120+0000","last_undegraded":"2024-09-18T01:04:48.342120+0000","last_fullsized":"2024-09-18T01:04:48.342120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:30.737729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15911,"num_read_kb":43615,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1b","version":"124'637","reported_seq":4489,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292326+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:04:48.292326+0000","last_peered":"2024-09-18T01:04:48.292326+0000","last_clean":"2024-09-18T01:04:48.292326+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:04:48.292326+0000","last_undegraded":"2024-09-18T01:04:48.292326+0000","last_fullsized":"2024-09-18T01:04:48.292326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1a","version":"125'240","reported_seq":6027,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342159+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:04:48.342159+0000","last_peered":"2024-09-18T01:04:48.342159+0000","last_clean":"2024-09-18T01:04:48.342159+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:04:48.342159+0000","last_undegraded":"2024-09-18T01:04:48.342159+0000","last_fullsized":"2024-09-18T01:04:48.342159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.19","version":"125'659","reported_seq":8713,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291043+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:04:48.291043+0000","last_peered":"2024-09-18T01:04:48.291043+0000","last_clean":"2024-09-18T01:04:48.291043+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:04:48.291043+0000","last_undegraded":"2024-09-18T01:04:48.291043+0000","last_fullsized":"2024-09-18T01:04:48.291043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.18","version":"124'740","reported_seq":8802,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327723+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:04:48.327723+0000","last_peered":"2024-09-18T01:04:48.327723+0000","last_clean":"2024-09-18T01:04:48.327723+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:04:48.327723+0000","last_undegraded":"2024-09-18T01:04:48.327723+0000","last_fullsized":"2024-09-18T01:04:48.327723+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.17","version":"124'903","reported_seq":8472,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292551+0000","last_change":"2024-09-18T01:04:46.431957+0000","last_active":"2024-09-18T01:04:48.292551+0000","last_peered":"2024-09-18T01:04:48.292551+0000","last_clean":"2024-09-18T01:04:48.292551+0000","last_became_active":"2024-09-18T01:04:41.398871+0000","last_became_peered":"2024-09-18T01:04:41.398871+0000","last_unstale":"2024-09-18T01:04:48.292551+0000","last_undegraded":"2024-09-18T01:04:48.292551+0000","last_fullsized":"2024-09-18T01:04:48.292551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:01:40.674921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.16","version":"125'364","reported_seq":3850,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290459+0000","last_change":"2024-09-18T01:04:46.501606+0000","last_active":"2024-09-18T01:04:48.290459+0000","last_peered":"2024-09-18T01:04:48.290459+0000","last_clean":"2024-09-18T01:04:48.290459+0000","last_became_active":"2024-09-18T01:03:38.521858+0000","last_became_peered":"2024-09-18T01:03:38.521858+0000","last_unstale":"2024-09-18T01:04:48.290459+0000","last_undegraded":"2024-09-18T01:04:48.290459+0000","last_fullsized":"2024-09-18T01:04:48.290459+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:28.415259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.15","version":"124'589","reported_seq":9875,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290721+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:04:48.290721+0000","last_peered":"2024-09-18T01:04:48.290721+0000","last_clean":"2024-09-18T01:04:48.290721+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:04:48.290721+0000","last_undegraded":"2024-09-18T01:04:48.290721+0000","last_fullsized":"2024-09-18T01:04:48.290721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.14","version":"122'578","reported_seq":5311,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290690+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:04:48.290690+0000","last_peered":"2024-09-18T01:04:48.290690+0000","last_clean":"2024-09-18T01:04:48.290690+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:04:48.290690+0000","last_undegraded":"2024-09-18T01:04:48.290690+0000","last_fullsized":"2024-09-18T01:04:48.290690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290877+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:04:48.290877+0000","last_peered":"2024-09-18T01:04:48.290877+0000","last_clean":"2024-09-18T01:04:48.290877+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:04:48.290877+0000","last_undegraded":"2024-09-18T01:04:48.290877+0000","last_fullsized":"2024-09-18T01:04:48.290877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7120,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292446+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:04:48.292446+0000","last_peered":"2024-09-18T01:04:48.292446+0000","last_clean":"2024-09-18T01:04:48.292446+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:04:48.292446+0000","last_undegraded":"2024-09-18T01:04:48.292446+0000","last_fullsized":"2024-09-18T01:04:48.292446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290889+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:04:48.290889+0000","last_peered":"2024-09-18T01:04:48.290889+0000","last_clean":"2024-09-18T01:04:48.290889+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:04:48.290889+0000","last_undegraded":"2024-09-18T01:04:48.290889+0000","last_fullsized":"2024-09-18T01:04:48.290889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10901,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327911+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:04:48.327911+0000","last_peered":"2024-09-18T01:04:48.327911+0000","last_clean":"2024-09-18T01:04:48.327911+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:04:48.327911+0000","last_undegraded":"2024-09-18T01:04:48.327911+0000","last_fullsized":"2024-09-18T01:04:48.327911+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290868+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:04:48.290868+0000","last_peered":"2024-09-18T01:04:48.290868+0000","last_clean":"2024-09-18T01:04:48.290868+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:04:48.290868+0000","last_undegraded":"2024-09-18T01:04:48.290868+0000","last_fullsized":"2024-09-18T01:04:48.290868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6330,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290646+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:04:48.290646+0000","last_peered":"2024-09-18T01:04:48.290646+0000","last_clean":"2024-09-18T01:04:48.290646+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:04:48.290646+0000","last_undegraded":"2024-09-18T01:04:48.290646+0000","last_fullsized":"2024-09-18T01:04:48.290646+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290695+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:04:48.290695+0000","last_peered":"2024-09-18T01:04:48.290695+0000","last_clean":"2024-09-18T01:04:48.290695+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:04:48.290695+0000","last_undegraded":"2024-09-18T01:04:48.290695+0000","last_fullsized":"2024-09-18T01:04:48.290695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8136,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327768+0000","last_change":"2024-09-18T01:04:46.428356+0000","last_active":"2024-09-18T01:04:48.327768+0000","last_peered":"2024-09-18T01:04:48.327768+0000","last_clean":"2024-09-18T01:04:48.327768+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:04:48.327768+0000","last_undegraded":"2024-09-18T01:04:48.327768+0000","last_fullsized":"2024-09-18T01:04:48.327768+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:58.414334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.3","version":"125'1081","reported_seq":7498,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342529+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:04:48.342529+0000","last_peered":"2024-09-18T01:04:48.342529+0000","last_clean":"2024-09-18T01:04:48.342529+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:04:48.342529+0000","last_undegraded":"2024-09-18T01:04:48.342529+0000","last_fullsized":"2024-09-18T01:04:48.342529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.2","version":"18'2","reported_seq":229,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291214+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:04:48.291214+0000","last_peered":"2024-09-18T01:04:48.291214+0000","last_clean":"2024-09-18T01:04:48.291214+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:04:48.291214+0000","last_undegraded":"2024-09-18T01:04:48.291214+0000","last_fullsized":"2024-09-18T01:04:48.291214+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6709,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290476+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:04:48.290476+0000","last_peered":"2024-09-18T01:04:48.290476+0000","last_clean":"2024-09-18T01:04:48.290476+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:04:48.290476+0000","last_undegraded":"2024-09-18T01:04:48.290476+0000","last_fullsized":"2024-09-18T01:04:48.290476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327618+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:04:48.327618+0000","last_peered":"2024-09-18T01:04:48.327618+0000","last_clean":"2024-09-18T01:04:48.327618+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:04:48.327618+0000","last_undegraded":"2024-09-18T01:04:48.327618+0000","last_fullsized":"2024-09-18T01:04:48.327618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4557,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292067+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:04:48.292067+0000","last_peered":"2024-09-18T01:04:48.292067+0000","last_clean":"2024-09-18T01:04:48.292067+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:04:48.292067+0000","last_undegraded":"2024-09-18T01:04:48.292067+0000","last_fullsized":"2024-09-18T01:04:48.292067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290289+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:04:48.290289+0000","last_peered":"2024-09-18T01:04:48.290289+0000","last_clean":"2024-09-18T01:04:48.290289+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:04:48.290289+0000","last_undegraded":"2024-09-18T01:04:48.290289+0000","last_fullsized":"2024-09-18T01:04:48.290289+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7001,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291198+0000","last_change":"2024-09-18T01:04:46.442975+0000","last_active":"2024-09-18T01:04:48.291198+0000","last_peered":"2024-09-18T01:04:48.291198+0000","last_clean":"2024-09-18T01:04:48.291198+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:04:48.291198+0000","last_undegraded":"2024-09-18T01:04:48.291198+0000","last_fullsized":"2024-09-18T01:04:48.291198+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:34:37.906968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.3","version":"16'1","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327750+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:04:48.327750+0000","last_peered":"2024-09-18T01:04:48.327750+0000","last_clean":"2024-09-18T01:04:48.327750+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:04:48.327750+0000","last_undegraded":"2024-09-18T01:04:48.327750+0000","last_fullsized":"2024-09-18T01:04:48.327750+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":390,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290309+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:04:48.290309+0000","last_peered":"2024-09-18T01:04:48.290309+0000","last_clean":"2024-09-18T01:04:48.290309+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:04:48.290309+0000","last_undegraded":"2024-09-18T01:04:48.290309+0000","last_fullsized":"2024-09-18T01:04:48.290309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8359,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327831+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:04:48.327831+0000","last_peered":"2024-09-18T01:04:48.327831+0000","last_clean":"2024-09-18T01:04:48.327831+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:04:48.327831+0000","last_undegraded":"2024-09-18T01:04:48.327831+0000","last_fullsized":"2024-09-18T01:04:48.327831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.9","version":"119'444","reported_seq":5074,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290397+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:04:48.290397+0000","last_peered":"2024-09-18T01:04:48.290397+0000","last_clean":"2024-09-18T01:04:48.290397+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:04:48.290397+0000","last_undegraded":"2024-09-18T01:04:48.290397+0000","last_fullsized":"2024-09-18T01:04:48.290397+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.a","version":"123'766","reported_seq":5421,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290728+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:04:48.290728+0000","last_peered":"2024-09-18T01:04:48.290728+0000","last_clean":"2024-09-18T01:04:48.290728+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:04:48.290728+0000","last_undegraded":"2024-09-18T01:04:48.290728+0000","last_fullsized":"2024-09-18T01:04:48.290728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.b","version":"114'511","reported_seq":6933,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290484+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:04:48.290484+0000","last_peered":"2024-09-18T01:04:48.290484+0000","last_clean":"2024-09-18T01:04:48.290484+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:04:48.290484+0000","last_undegraded":"2024-09-18T01:04:48.290484+0000","last_fullsized":"2024-09-18T01:04:48.290484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.c","version":"114'371","reported_seq":6038,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292358+0000","last_change":"2024-09-18T01:04:46.431127+0000","last_active":"2024-09-18T01:04:48.292358+0000","last_peered":"2024-09-18T01:04:48.292358+0000","last_clean":"2024-09-18T01:04:48.292358+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:04:48.292358+0000","last_undegraded":"2024-09-18T01:04:48.292358+0000","last_fullsized":"2024-09-18T01:04:48.292358+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:32.986756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.d","version":"120'530","reported_seq":6174,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290723+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:04:48.290723+0000","last_peered":"2024-09-18T01:04:48.290723+0000","last_clean":"2024-09-18T01:04:48.290723+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:04:48.290723+0000","last_undegraded":"2024-09-18T01:04:48.290723+0000","last_fullsized":"2024-09-18T01:04:48.290723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.e","version":"125'399","reported_seq":6857,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342145+0000","last_change":"2024-09-18T01:04:46.426778+0000","last_active":"2024-09-18T01:04:48.342145+0000","last_peered":"2024-09-18T01:04:48.342145+0000","last_clean":"2024-09-18T01:04:48.342145+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:04:48.342145+0000","last_undegraded":"2024-09-18T01:04:48.342145+0000","last_fullsized":"2024-09-18T01:04:48.342145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:26.248994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.f","version":"119'614","reported_seq":13234,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290466+0000","last_change":"2024-09-18T01:04:46.501917+0000","last_active":"2024-09-18T01:04:48.290466+0000","last_peered":"2024-09-18T01:04:48.290466+0000","last_clean":"2024-09-18T01:04:48.290466+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:04:48.290466+0000","last_undegraded":"2024-09-18T01:04:48.290466+0000","last_fullsized":"2024-09-18T01:04:48.290466+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:30.243671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017582590000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.10","version":"122'439","reported_seq":4598,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291887+0000","last_change":"2024-09-18T01:04:46.430699+0000","last_active":"2024-09-18T01:04:48.291887+0000","last_peered":"2024-09-18T01:04:48.291887+0000","last_clean":"2024-09-18T01:04:48.291887+0000","last_became_active":"2024-09-18T01:04:39.295254+0000","last_became_peered":"2024-09-18T01:04:39.295254+0000","last_unstale":"2024-09-18T01:04:48.291887+0000","last_undegraded":"2024-09-18T01:04:48.291887+0000","last_fullsized":"2024-09-18T01:04:48.291887+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:40:54.320305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.11","version":"121'408","reported_seq":4490,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290611+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:04:48.290611+0000","last_peered":"2024-09-18T01:04:48.290611+0000","last_clean":"2024-09-18T01:04:48.290611+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:04:48.290611+0000","last_undegraded":"2024-09-18T01:04:48.290611+0000","last_fullsized":"2024-09-18T01:04:48.290611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.12","version":"122'976","reported_seq":10461,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292009+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:04:48.292009+0000","last_peered":"2024-09-18T01:04:48.292009+0000","last_clean":"2024-09-18T01:04:48.292009+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:04:48.292009+0000","last_undegraded":"2024-09-18T01:04:48.292009+0000","last_fullsized":"2024-09-18T01:04:48.292009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.13","version":"124'313","reported_seq":3902,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290409+0000","last_change":"2024-09-18T01:04:46.501704+0000","last_active":"2024-09-18T01:04:48.290409+0000","last_peered":"2024-09-18T01:04:48.290409+0000","last_clean":"2024-09-18T01:04:48.290409+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:04:48.290409+0000","last_undegraded":"2024-09-18T01:04:48.290409+0000","last_fullsized":"2024-09-18T01:04:48.290409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:21.447375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265612,"num_read_kb":1200487,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1358,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640287,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640289,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46564,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325276,"statfs":{"total":96636764160,"available":96589082624,"internally_reserved":0,"allocated":1515520,"data_stored":624298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640288,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1568768,"data_stored":1214647,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640287,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77084,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294756,"statfs":{"total":96636764160,"available":96557830144,"internally_reserved":0,"allocated":1511424,"data_stored":624279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64620,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307220,"statfs":{"total":96636764160,"available":96570593280,"internally_reserved":0,"allocated":1519616,"data_stored":624317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46616,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325224,"statfs":{"total":96636764160,"available":96589029376,"internally_reserved":0,"allocated":1572864,"data_stored":1214666,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:04:51.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:04:52.099 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:04:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:04:52.112 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-18T01:04:51.446441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265729,"num_read_kb":1200588,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1360,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":281492,"kb_used_data":7508,"kb_used_omap":7,"kb_used_meta":273848,"kb_avail":471577708,"statfs":{"total":483183820800,"available":482895572992,"internally_reserved":0,"allocated":7688192,"data_stored":4302207,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":7973,"internal_metadata":280420571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-65294183,"num_objects":-45,"num_object_clones":-6,"num_object_copies":-90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-45,"num_whiteouts":-5,"num_read":577,"num_read_kb":501,"num_write":132,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017991"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10515,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292565+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:04:48.292565+0000","last_peered":"2024-09-18T01:04:48.292565+0000","last_clean":"2024-09-18T01:04:48.292565+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:04:48.292565+0000","last_undegraded":"2024-09-18T01:04:48.292565+0000","last_fullsized":"2024-09-18T01:04:48.292565+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1e","version":"114'1473","reported_seq":7711,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327848+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:04:48.327848+0000","last_peered":"2024-09-18T01:04:48.327848+0000","last_clean":"2024-09-18T01:04:48.327848+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:04:48.327848+0000","last_undegraded":"2024-09-18T01:04:48.327848+0000","last_fullsized":"2024-09-18T01:04:48.327848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1d","version":"124'455","reported_seq":7427,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342121+0000","last_change":"2024-09-18T01:04:46.426329+0000","last_active":"2024-09-18T01:04:48.342121+0000","last_peered":"2024-09-18T01:04:48.342121+0000","last_clean":"2024-09-18T01:04:48.342121+0000","last_became_active":"2024-09-18T01:02:49.296093+0000","last_became_peered":"2024-09-18T01:02:49.296093+0000","last_unstale":"2024-09-18T01:04:48.342121+0000","last_undegraded":"2024-09-18T01:04:48.342121+0000","last_fullsized":"2024-09-18T01:04:48.342121+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:42.667965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000500068,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1c","version":"125'869","reported_seq":12779,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342120+0000","last_change":"2024-09-18T01:04:46.427109+0000","last_active":"2024-09-18T01:04:48.342120+0000","last_peered":"2024-09-18T01:04:48.342120+0000","last_clean":"2024-09-18T01:04:48.342120+0000","last_became_active":"2024-09-18T01:02:49.295898+0000","last_became_peered":"2024-09-18T01:02:49.295898+0000","last_unstale":"2024-09-18T01:04:48.342120+0000","last_undegraded":"2024-09-18T01:04:48.342120+0000","last_fullsized":"2024-09-18T01:04:48.342120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:30.737729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15911,"num_read_kb":43615,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1b","version":"124'637","reported_seq":4489,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292326+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:04:48.292326+0000","last_peered":"2024-09-18T01:04:48.292326+0000","last_clean":"2024-09-18T01:04:48.292326+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:04:48.292326+0000","last_undegraded":"2024-09-18T01:04:48.292326+0000","last_fullsized":"2024-09-18T01:04:48.292326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.1a","version":"125'240","reported_seq":6027,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342159+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:04:48.342159+0000","last_peered":"2024-09-18T01:04:48.342159+0000","last_clean":"2024-09-18T01:04:48.342159+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:04:48.342159+0000","last_undegraded":"2024-09-18T01:04:48.342159+0000","last_fullsized":"2024-09-18T01:04:48.342159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.19","version":"125'659","reported_seq":8713,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291043+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:04:48.291043+0000","last_peered":"2024-09-18T01:04:48.291043+0000","last_clean":"2024-09-18T01:04:48.291043+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:04:48.291043+0000","last_undegraded":"2024-09-18T01:04:48.291043+0000","last_fullsized":"2024-09-18T01:04:48.291043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.18","version":"124'740","reported_seq":8802,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327723+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:04:48.327723+0000","last_peered":"2024-09-18T01:04:48.327723+0000","last_clean":"2024-09-18T01:04:48.327723+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:04:48.327723+0000","last_undegraded":"2024-09-18T01:04:48.327723+0000","last_fullsized":"2024-09-18T01:04:48.327723+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.17","version":"124'903","reported_seq":8472,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292551+0000","last_change":"2024-09-18T01:04:46.431957+0000","last_active":"2024-09-18T01:04:48.292551+0000","last_peered":"2024-09-18T01:04:48.292551+0000","last_clean":"2024-09-18T01:04:48.292551+0000","last_became_active":"2024-09-18T01:04:41.398871+0000","last_became_peered":"2024-09-18T01:04:41.398871+0000","last_unstale":"2024-09-18T01:04:48.292551+0000","last_undegraded":"2024-09-18T01:04:48.292551+0000","last_fullsized":"2024-09-18T01:04:48.292551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:01:40.674921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.16","version":"125'364","reported_seq":3850,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290459+0000","last_change":"2024-09-18T01:04:46.501606+0000","last_active":"2024-09-18T01:04:48.290459+0000","last_peered":"2024-09-18T01:04:48.290459+0000","last_clean":"2024-09-18T01:04:48.290459+0000","last_became_active":"2024-09-18T01:03:38.521858+0000","last_became_peered":"2024-09-18T01:03:38.521858+0000","last_unstale":"2024-09-18T01:04:48.290459+0000","last_undegraded":"2024-09-18T01:04:48.290459+0000","last_fullsized":"2024-09-18T01:04:48.290459+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:28.415259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.15","version":"124'589","reported_seq":9875,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290721+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:04:48.290721+0000","last_peered":"2024-09-18T01:04:48.290721+0000","last_clean":"2024-09-18T01:04:48.290721+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:04:48.290721+0000","last_undegraded":"2024-09-18T01:04:48.290721+0000","last_fullsized":"2024-09-18T01:04:48.290721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.14","version":"122'578","reported_seq":5311,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290690+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:04:48.290690+0000","last_peered":"2024-09-18T01:04:48.290690+0000","last_clean":"2024-09-18T01:04:48.290690+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:04:48.290690+0000","last_undegraded":"2024-09-18T01:04:48.290690+0000","last_fullsized":"2024-09-18T01:04:48.290690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290877+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:04:48.290877+0000","last_peered":"2024-09-18T01:04:48.290877+0000","last_clean":"2024-09-18T01:04:48.290877+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:04:48.290877+0000","last_undegraded":"2024-09-18T01:04:48.290877+0000","last_fullsized":"2024-09-18T01:04:48.290877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7120,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292446+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:04:48.292446+0000","last_peered":"2024-09-18T01:04:48.292446+0000","last_clean":"2024-09-18T01:04:48.292446+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:04:48.292446+0000","last_undegraded":"2024-09-18T01:04:48.292446+0000","last_fullsized":"2024-09-18T01:04:48.292446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290889+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:04:48.290889+0000","last_peered":"2024-09-18T01:04:48.290889+0000","last_clean":"2024-09-18T01:04:48.290889+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:04:48.290889+0000","last_undegraded":"2024-09-18T01:04:48.290889+0000","last_fullsized":"2024-09-18T01:04:48.290889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10901,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327911+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:04:48.327911+0000","last_peered":"2024-09-18T01:04:48.327911+0000","last_clean":"2024-09-18T01:04:48.327911+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:04:48.327911+0000","last_undegraded":"2024-09-18T01:04:48.327911+0000","last_fullsized":"2024-09-18T01:04:48.327911+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290868+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:04:48.290868+0000","last_peered":"2024-09-18T01:04:48.290868+0000","last_clean":"2024-09-18T01:04:48.290868+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:04:48.290868+0000","last_undegraded":"2024-09-18T01:04:48.290868+0000","last_fullsized":"2024-09-18T01:04:48.290868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6330,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290646+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:04:48.290646+0000","last_peered":"2024-09-18T01:04:48.290646+0000","last_clean":"2024-09-18T01:04:48.290646+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:04:48.290646+0000","last_undegraded":"2024-09-18T01:04:48.290646+0000","last_fullsized":"2024-09-18T01:04:48.290646+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290695+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:04:48.290695+0000","last_peered":"2024-09-18T01:04:48.290695+0000","last_clean":"2024-09-18T01:04:48.290695+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:04:48.290695+0000","last_undegraded":"2024-09-18T01:04:48.290695+0000","last_fullsized":"2024-09-18T01:04:48.290695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8136,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327768+0000","last_change":"2024-09-18T01:04:46.428356+0000","last_active":"2024-09-18T01:04:48.327768+0000","last_peered":"2024-09-18T01:04:48.327768+0000","last_clean":"2024-09-18T01:04:48.327768+0000","last_became_active":"2024-09-18T01:02:49.293995+0000","last_became_peered":"2024-09-18T01:02:49.293995+0000","last_unstale":"2024-09-18T01:04:48.327768+0000","last_undegraded":"2024-09-18T01:04:48.327768+0000","last_fullsized":"2024-09-18T01:04:48.327768+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:58.414334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.3","version":"125'1081","reported_seq":7498,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342529+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:04:48.342529+0000","last_peered":"2024-09-18T01:04:48.342529+0000","last_clean":"2024-09-18T01:04:48.342529+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:04:48.342529+0000","last_undegraded":"2024-09-18T01:04:48.342529+0000","last_fullsized":"2024-09-18T01:04:48.342529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.2","version":"18'2","reported_seq":229,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291214+0000","last_change":"2024-09-18T01:02:49.329451+0000","last_active":"2024-09-18T01:04:48.291214+0000","last_peered":"2024-09-18T01:04:48.291214+0000","last_clean":"2024-09-18T01:04:48.291214+0000","last_became_active":"2024-09-18T01:02:49.293240+0000","last_became_peered":"2024-09-18T01:02:49.293240+0000","last_unstale":"2024-09-18T01:04:48.291214+0000","last_undegraded":"2024-09-18T01:04:48.291214+0000","last_fullsized":"2024-09-18T01:04:48.291214+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:05.306836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6709,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290476+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:04:48.290476+0000","last_peered":"2024-09-18T01:04:48.290476+0000","last_clean":"2024-09-18T01:04:48.290476+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:04:48.290476+0000","last_undegraded":"2024-09-18T01:04:48.290476+0000","last_fullsized":"2024-09-18T01:04:48.290476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327618+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:04:48.327618+0000","last_peered":"2024-09-18T01:04:48.327618+0000","last_clean":"2024-09-18T01:04:48.327618+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:04:48.327618+0000","last_undegraded":"2024-09-18T01:04:48.327618+0000","last_fullsized":"2024-09-18T01:04:48.327618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4557,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292067+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:04:48.292067+0000","last_peered":"2024-09-18T01:04:48.292067+0000","last_clean":"2024-09-18T01:04:48.292067+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:04:48.292067+0000","last_undegraded":"2024-09-18T01:04:48.292067+0000","last_fullsized":"2024-09-18T01:04:48.292067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290289+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:04:48.290289+0000","last_peered":"2024-09-18T01:04:48.290289+0000","last_clean":"2024-09-18T01:04:48.290289+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:04:48.290289+0000","last_undegraded":"2024-09-18T01:04:48.290289+0000","last_fullsized":"2024-09-18T01:04:48.290289+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7001,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291198+0000","last_change":"2024-09-18T01:04:46.442975+0000","last_active":"2024-09-18T01:04:48.291198+0000","last_peered":"2024-09-18T01:04:48.291198+0000","last_clean":"2024-09-18T01:04:48.291198+0000","last_became_active":"2024-09-18T01:02:49.294499+0000","last_became_peered":"2024-09-18T01:02:49.294499+0000","last_unstale":"2024-09-18T01:04:48.291198+0000","last_undegraded":"2024-09-18T01:04:48.291198+0000","last_fullsized":"2024-09-18T01:04:48.291198+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:34:37.906968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"2.3","version":"16'1","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327750+0000","last_change":"2024-09-18T01:02:49.307144+0000","last_active":"2024-09-18T01:04:48.327750+0000","last_peered":"2024-09-18T01:04:48.327750+0000","last_clean":"2024-09-18T01:04:48.327750+0000","last_became_active":"2024-09-18T01:02:49.296021+0000","last_became_peered":"2024-09-18T01:02:49.296021+0000","last_unstale":"2024-09-18T01:04:48.327750+0000","last_undegraded":"2024-09-18T01:04:48.327750+0000","last_fullsized":"2024-09-18T01:04:48.327750+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:32:36.709073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":390,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290309+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:04:48.290309+0000","last_peered":"2024-09-18T01:04:48.290309+0000","last_clean":"2024-09-18T01:04:48.290309+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:04:48.290309+0000","last_undegraded":"2024-09-18T01:04:48.290309+0000","last_fullsized":"2024-09-18T01:04:48.290309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8359,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.327831+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:04:48.327831+0000","last_peered":"2024-09-18T01:04:48.327831+0000","last_clean":"2024-09-18T01:04:48.327831+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:04:48.327831+0000","last_undegraded":"2024-09-18T01:04:48.327831+0000","last_fullsized":"2024-09-18T01:04:48.327831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.9","version":"119'444","reported_seq":5074,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290397+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:04:48.290397+0000","last_peered":"2024-09-18T01:04:48.290397+0000","last_clean":"2024-09-18T01:04:48.290397+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:04:48.290397+0000","last_undegraded":"2024-09-18T01:04:48.290397+0000","last_fullsized":"2024-09-18T01:04:48.290397+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.a","version":"123'766","reported_seq":5421,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290728+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:04:48.290728+0000","last_peered":"2024-09-18T01:04:48.290728+0000","last_clean":"2024-09-18T01:04:48.290728+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:04:48.290728+0000","last_undegraded":"2024-09-18T01:04:48.290728+0000","last_fullsized":"2024-09-18T01:04:48.290728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.b","version":"114'511","reported_seq":6933,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290484+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:04:48.290484+0000","last_peered":"2024-09-18T01:04:48.290484+0000","last_clean":"2024-09-18T01:04:48.290484+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:04:48.290484+0000","last_undegraded":"2024-09-18T01:04:48.290484+0000","last_fullsized":"2024-09-18T01:04:48.290484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.c","version":"114'371","reported_seq":6038,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292358+0000","last_change":"2024-09-18T01:04:46.431127+0000","last_active":"2024-09-18T01:04:48.292358+0000","last_peered":"2024-09-18T01:04:48.292358+0000","last_clean":"2024-09-18T01:04:48.292358+0000","last_became_active":"2024-09-18T01:02:49.291504+0000","last_became_peered":"2024-09-18T01:02:49.291504+0000","last_unstale":"2024-09-18T01:04:48.292358+0000","last_undegraded":"2024-09-18T01:04:48.292358+0000","last_fullsized":"2024-09-18T01:04:48.292358+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:32.986756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.d","version":"120'530","reported_seq":6174,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290723+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:04:48.290723+0000","last_peered":"2024-09-18T01:04:48.290723+0000","last_clean":"2024-09-18T01:04:48.290723+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:04:48.290723+0000","last_undegraded":"2024-09-18T01:04:48.290723+0000","last_fullsized":"2024-09-18T01:04:48.290723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.e","version":"125'399","reported_seq":6857,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.342145+0000","last_change":"2024-09-18T01:04:46.426778+0000","last_active":"2024-09-18T01:04:48.342145+0000","last_peered":"2024-09-18T01:04:48.342145+0000","last_clean":"2024-09-18T01:04:48.342145+0000","last_became_active":"2024-09-18T01:02:49.295985+0000","last_became_peered":"2024-09-18T01:02:49.295985+0000","last_unstale":"2024-09-18T01:04:48.342145+0000","last_undegraded":"2024-09-18T01:04:48.342145+0000","last_fullsized":"2024-09-18T01:04:48.342145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:20:26.248994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.f","version":"119'614","reported_seq":13234,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290466+0000","last_change":"2024-09-18T01:04:46.501917+0000","last_active":"2024-09-18T01:04:48.290466+0000","last_peered":"2024-09-18T01:04:48.290466+0000","last_clean":"2024-09-18T01:04:48.290466+0000","last_became_active":"2024-09-18T01:02:49.290971+0000","last_became_peered":"2024-09-18T01:02:49.290971+0000","last_unstale":"2024-09-18T01:04:48.290466+0000","last_undegraded":"2024-09-18T01:04:48.290466+0000","last_fullsized":"2024-09-18T01:04:48.290466+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:30.243671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017582590000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.10","version":"122'439","reported_seq":4598,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.291887+0000","last_change":"2024-09-18T01:04:46.430699+0000","last_active":"2024-09-18T01:04:48.291887+0000","last_peered":"2024-09-18T01:04:48.291887+0000","last_clean":"2024-09-18T01:04:48.291887+0000","last_became_active":"2024-09-18T01:04:39.295254+0000","last_became_peered":"2024-09-18T01:04:39.295254+0000","last_unstale":"2024-09-18T01:04:48.291887+0000","last_undegraded":"2024-09-18T01:04:48.291887+0000","last_fullsized":"2024-09-18T01:04:48.291887+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:40:54.320305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.11","version":"121'408","reported_seq":4490,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290611+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:04:48.290611+0000","last_peered":"2024-09-18T01:04:48.290611+0000","last_clean":"2024-09-18T01:04:48.290611+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:04:48.290611+0000","last_undegraded":"2024-09-18T01:04:48.290611+0000","last_fullsized":"2024-09-18T01:04:48.290611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.12","version":"122'976","reported_seq":10461,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.292009+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:04:48.292009+0000","last_peered":"2024-09-18T01:04:48.292009+0000","last_clean":"2024-09-18T01:04:48.292009+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:04:48.292009+0000","last_undegraded":"2024-09-18T01:04:48.292009+0000","last_fullsized":"2024-09-18T01:04:48.292009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"3"}]},{"pgid":"3.13","version":"124'313","reported_seq":3902,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-18T01:04:48.290409+0000","last_change":"2024-09-18T01:04:46.501704+0000","last_active":"2024-09-18T01:04:48.290409+0000","last_peered":"2024-09-18T01:04:48.290409+0000","last_clean":"2024-09-18T01:04:48.290409+0000","last_became_active":"2024-09-18T01:02:49.293063+0000","last_became_peered":"2024-09-18T01:02:49.293063+0000","last_unstale":"2024-09-18T01:04:48.290409+0000","last_undegraded":"2024-09-18T01:04:48.290409+0000","last_fullsized":"2024-09-18T01:04:48.290409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:21.447375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265612,"num_read_kb":1200487,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1358,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":0,"seq":47244640287,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640289,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46564,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325276,"statfs":{"total":96636764160,"available":96589082624,"internally_reserved":0,"allocated":1515520,"data_stored":624298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640288,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1568768,"data_stored":1214647,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640287,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77084,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294756,"statfs":{"total":96636764160,"available":96557830144,"internally_reserved":0,"allocated":1511424,"data_stored":624279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64620,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307220,"statfs":{"total":96636764160,"available":96570593280,"internally_reserved":0,"allocated":1519616,"data_stored":624317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46616,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325224,"statfs":{"total":96636764160,"available":96589029376,"internally_reserved":0,"allocated":1572864,"data_stored":1214666,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:04:52.114 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:04:52.114 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T01:04:52.114 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-18T01:04:52.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-18T01:04:52.411 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-18T01:04:52.409+0000 7fdf07c7e640 -1 mon.a@0(leader).osd e130 definitely_dead 0 2024-09-18T01:04:52.448 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.5. 2024-09-18T01:04:52.465 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-18T01:04:57.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-18T01:04:57.595 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-18T01:04:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:04:57.605 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-18T01:04:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:04:57.614 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-18T01:04:57.614 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T01:04:57.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T01:04:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T01:04:58.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T01:04:59.316 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T01:04:59.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T01:05:00.322 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T01:05:00.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-18T01:05:01.345 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-18T01:05:01.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T01:05:02.346 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T01:05:02.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-18T01:05:03.352 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-18T01:05:03.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T01:05:04.351 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T01:05:04.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T01:05:05.355 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T01:05:05.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T01:05:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T01:05:06.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T01:05:07.378 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T01:05:07.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T01:05:08.385 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T01:05:08.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T01:05:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T01:05:09.411 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-18T01:05:09.411 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-18T01:05:09.411 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-18T01:05:09.412 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-18T01:05:09.412 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T01:05:09.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:05:09.711 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:09.712 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":148,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:05:09.391503+0000","last_up_change":"2024-09-18T01:02:24.290620+0000","last_in_change":"2024-09-18T01:04:52.444874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:02:24.388064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:02:30.676199+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T01:02:40.390732+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6817","nonce":1626166637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6819","nonce":1626166637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6823","nonce":1626166637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6821","nonce":1626166637}]},"public_addr":"172.21.15.31:6817/1626166637","cluster_addr":"172.21.15.31:6819/1626166637","heartbeat_back_addr":"172.21.15.31:6823/1626166637","heartbeat_front_addr":"172.21.15.31:6821/1626166637","state":["exists","up"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6801","nonce":2210970259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6803","nonce":2210970259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6807","nonce":2210970259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6805","nonce":2210970259}]},"public_addr":"172.21.15.31:6801/2210970259","cluster_addr":"172.21.15.31:6803/2210970259","heartbeat_back_addr":"172.21.15.31:6807/2210970259","heartbeat_front_addr":"172.21.15.31:6805/2210970259","state":["exists","up"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6809","nonce":617133675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6811","nonce":617133675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6815","nonce":617133675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6813","nonce":617133675}]},"public_addr":"172.21.15.31:6809/617133675","cluster_addr":"172.21.15.31:6811/617133675","heartbeat_back_addr":"172.21.15.31:6815/617133675","heartbeat_front_addr":"172.21.15.31:6813/617133675","state":["exists","up"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6809","nonce":151031464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6811","nonce":151031464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6815","nonce":151031464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6813","nonce":151031464}]},"public_addr":"172.21.15.98:6809/151031464","cluster_addr":"172.21.15.98:6811/151031464","heartbeat_back_addr":"172.21.15.98:6815/151031464","heartbeat_front_addr":"172.21.15.98:6813/151031464","state":["exists","up"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6817","nonce":1499036965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6819","nonce":1499036965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6823","nonce":1499036965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6821","nonce":1499036965}]},"public_addr":"172.21.15.98:6817/1499036965","cluster_addr":"172.21.15.98:6819/1499036965","heartbeat_back_addr":"172.21.15.98:6823/1499036965","heartbeat_front_addr":"172.21.15.98:6821/1499036965","state":["exists","up"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6801","nonce":2310498524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6803","nonce":2310498524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6807","nonce":2310498524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6805","nonce":2310498524}]},"public_addr":"172.21.15.98:6801/2310498524","cluster_addr":"172.21.15.98:6803/2310498524","heartbeat_back_addr":"172.21.15.98:6807/2310498524","heartbeat_front_addr":"172.21.15.98:6805/2310498524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.877700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.360930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.403205+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.952123+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:22.099545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.694132+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[1,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:05:09.724 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T01:05:09.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:05:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":148,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:05:09.391503+0000","last_up_change":"2024-09-18T01:02:24.290620+0000","last_in_change":"2024-09-18T01:04:52.444874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:02:24.388064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:02:30.676199+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T01:02:40.390732+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6817","nonce":1626166637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6819","nonce":1626166637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6823","nonce":1626166637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6821","nonce":1626166637}]},"public_addr":"172.21.15.31:6817/1626166637","cluster_addr":"172.21.15.31:6819/1626166637","heartbeat_back_addr":"172.21.15.31:6823/1626166637","heartbeat_front_addr":"172.21.15.31:6821/1626166637","state":["exists","up"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6801","nonce":2210970259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6803","nonce":2210970259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6807","nonce":2210970259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6805","nonce":2210970259}]},"public_addr":"172.21.15.31:6801/2210970259","cluster_addr":"172.21.15.31:6803/2210970259","heartbeat_back_addr":"172.21.15.31:6807/2210970259","heartbeat_front_addr":"172.21.15.31:6805/2210970259","state":["exists","up"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6809","nonce":617133675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6811","nonce":617133675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6815","nonce":617133675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6813","nonce":617133675}]},"public_addr":"172.21.15.31:6809/617133675","cluster_addr":"172.21.15.31:6811/617133675","heartbeat_back_addr":"172.21.15.31:6815/617133675","heartbeat_front_addr":"172.21.15.31:6813/617133675","state":["exists","up"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6809","nonce":151031464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6811","nonce":151031464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6815","nonce":151031464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6813","nonce":151031464}]},"public_addr":"172.21.15.98:6809/151031464","cluster_addr":"172.21.15.98:6811/151031464","heartbeat_back_addr":"172.21.15.98:6815/151031464","heartbeat_front_addr":"172.21.15.98:6813/151031464","state":["exists","up"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6817","nonce":1499036965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6819","nonce":1499036965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6823","nonce":1499036965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6821","nonce":1499036965}]},"public_addr":"172.21.15.98:6817/1499036965","cluster_addr":"172.21.15.98:6819/1499036965","heartbeat_back_addr":"172.21.15.98:6823/1499036965","heartbeat_front_addr":"172.21.15.98:6821/1499036965","state":["exists","up"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6801","nonce":2310498524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6803","nonce":2310498524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6807","nonce":2310498524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6805","nonce":2310498524}]},"public_addr":"172.21.15.98:6801/2310498524","cluster_addr":"172.21.15.98:6803/2310498524","heartbeat_back_addr":"172.21.15.98:6807/2310498524","heartbeat_front_addr":"172.21.15.98:6805/2310498524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.877700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.360930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.403205+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.952123+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:22.099545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.694132+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[1,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:05:10.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T01:05:10.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T01:05:10.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T01:05:10.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T01:05:10.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T01:05:10.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T01:05:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:47244640292 2024-09-18T01:05:10.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T01:05:10.232 INFO:teuthology.orchestra.run.smithi031.stdout:47244640294 2024-09-18T01:05:10.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T01:05:10.235 INFO:teuthology.orchestra.run.smithi031.stdout:47244640292 2024-09-18T01:05:10.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T01:05:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:47244640294 2024-09-18T01:05:10.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T01:05:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:47244640293 2024-09-18T01:05:10.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T01:05:10.411 INFO:teuthology.orchestra.run.smithi031.stdout:47244640293 2024-09-18T01:05:10.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:05:10.550 INFO:teuthology.orchestra.run.smithi031.stdout:47244640292 2024-09-18T01:05:10.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640292 got 47244640292 for osd.2 2024-09-18T01:05:10.564 DEBUG:teuthology.parallel:result is None 2024-09-18T01:05:10.716 INFO:teuthology.orchestra.run.smithi031.stdout:47244640292 2024-09-18T01:05:10.725 INFO:teuthology.orchestra.run.smithi031.stdout:47244640294 2024-09-18T01:05:10.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640293 got 47244640292 for osd.0 2024-09-18T01:05:10.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640294 got 47244640294 for osd.1 2024-09-18T01:05:10.740 DEBUG:teuthology.parallel:result is None 2024-09-18T01:05:10.742 INFO:teuthology.orchestra.run.smithi031.stdout:47244640292 2024-09-18T01:05:10.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640292 got 47244640292 for osd.5 2024-09-18T01:05:10.760 DEBUG:teuthology.parallel:result is None 2024-09-18T01:05:10.780 INFO:teuthology.orchestra.run.smithi031.stdout:47244640294 2024-09-18T01:05:10.782 INFO:teuthology.orchestra.run.smithi031.stdout:47244640293 2024-09-18T01:05:10.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640294 got 47244640294 for osd.4 2024-09-18T01:05:10.794 DEBUG:teuthology.parallel:result is None 2024-09-18T01:05:10.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640293 got 47244640293 for osd.3 2024-09-18T01:05:10.795 DEBUG:teuthology.parallel:result is None 2024-09-18T01:05:11.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:05:12.035 INFO:teuthology.orchestra.run.smithi031.stdout:47244640292 2024-09-18T01:05:12.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640293 got 47244640292 for osd.0 2024-09-18T01:05:13.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T01:05:13.514 INFO:teuthology.orchestra.run.smithi031.stdout:47244640293 2024-09-18T01:05:13.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640293 got 47244640293 for osd.0 2024-09-18T01:05:13.528 DEBUG:teuthology.parallel:result is None 2024-09-18T01:05:13.528 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T01:05:13.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:05:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:13.821 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:05:13.832 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-18T01:05:12.371328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265726,"num_read_kb":1200585,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":323956,"kb_used_data":10460,"kb_used_omap":9,"kb_used_meta":313334,"kb_avail":565907084,"statfs":{"total":579820584960,"available":579488854016,"internally_reserved":0,"allocated":10711040,"data_stored":5548584,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":320854688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001394"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10557,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396239+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:05:09.396239+0000","last_peered":"2024-09-18T01:05:09.396239+0000","last_clean":"2024-09-18T01:05:09.396239+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:05:09.396239+0000","last_undegraded":"2024-09-18T01:05:09.396239+0000","last_fullsized":"2024-09-18T01:05:09.396239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'1473","reported_seq":7752,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929303+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:05:09.929303+0000","last_peered":"2024-09-18T01:05:09.929303+0000","last_clean":"2024-09-18T01:05:09.929303+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:05:09.929303+0000","last_undegraded":"2024-09-18T01:05:09.929303+0000","last_fullsized":"2024-09-18T01:05:09.929303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"124'455","reported_seq":7452,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805024+0000","last_change":"2024-09-18T01:04:56.309151+0000","last_active":"2024-09-18T01:05:09.805024+0000","last_peered":"2024-09-18T01:05:09.805024+0000","last_clean":"2024-09-18T01:05:09.805024+0000","last_became_active":"2024-09-18T01:04:56.307394+0000","last_became_peered":"2024-09-18T01:04:56.307394+0000","last_unstale":"2024-09-18T01:05:09.805024+0000","last_undegraded":"2024-09-18T01:05:09.805024+0000","last_fullsized":"2024-09-18T01:05:09.805024+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:35.044114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020314901999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"125'869","reported_seq":12809,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804894+0000","last_change":"2024-09-18T01:04:57.305367+0000","last_active":"2024-09-18T01:05:09.804894+0000","last_peered":"2024-09-18T01:05:09.804894+0000","last_clean":"2024-09-18T01:05:09.804894+0000","last_became_active":"2024-09-18T01:04:57.303594+0000","last_became_peered":"2024-09-18T01:04:57.303594+0000","last_unstale":"2024-09-18T01:05:09.804894+0000","last_undegraded":"2024-09-18T01:05:09.804894+0000","last_fullsized":"2024-09-18T01:05:09.804894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:51.883218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15908,"num_read_kb":43612,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"124'637","reported_seq":4531,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395893+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:05:09.395893+0000","last_peered":"2024-09-18T01:05:09.395893+0000","last_clean":"2024-09-18T01:05:09.395893+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:05:09.395893+0000","last_undegraded":"2024-09-18T01:05:09.395893+0000","last_fullsized":"2024-09-18T01:05:09.395893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"125'240","reported_seq":6067,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397675+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:05:09.397675+0000","last_peered":"2024-09-18T01:05:09.397675+0000","last_clean":"2024-09-18T01:05:09.397675+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:05:09.397675+0000","last_undegraded":"2024-09-18T01:05:09.397675+0000","last_fullsized":"2024-09-18T01:05:09.397675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"125'659","reported_seq":8756,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395703+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:05:09.395703+0000","last_peered":"2024-09-18T01:05:09.395703+0000","last_clean":"2024-09-18T01:05:09.395703+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:05:09.395703+0000","last_undegraded":"2024-09-18T01:05:09.395703+0000","last_fullsized":"2024-09-18T01:05:09.395703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"124'740","reported_seq":8844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"124'903","reported_seq":8734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396311+0000","last_change":"2024-09-18T01:04:57.304386+0000","last_active":"2024-09-18T01:05:09.396311+0000","last_peered":"2024-09-18T01:05:09.396311+0000","last_clean":"2024-09-18T01:05:09.396311+0000","last_became_active":"2024-09-18T01:04:57.302866+0000","last_became_peered":"2024-09-18T01:04:57.302866+0000","last_unstale":"2024-09-18T01:05:09.396311+0000","last_undegraded":"2024-09-18T01:05:09.396311+0000","last_fullsized":"2024-09-18T01:05:09.396311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:17.405000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"125'364","reported_seq":3882,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805054+0000","last_change":"2024-09-18T01:04:56.305126+0000","last_active":"2024-09-18T01:05:09.805054+0000","last_peered":"2024-09-18T01:05:09.805054+0000","last_clean":"2024-09-18T01:05:09.805054+0000","last_became_active":"2024-09-18T01:04:56.303131+0000","last_became_peered":"2024-09-18T01:04:56.303131+0000","last_unstale":"2024-09-18T01:05:09.805054+0000","last_undegraded":"2024-09-18T01:05:09.805054+0000","last_fullsized":"2024-09-18T01:05:09.805054+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:10.930635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"124'589","reported_seq":9916,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389745+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:05:08.389745+0000","last_peered":"2024-09-18T01:05:08.389745+0000","last_clean":"2024-09-18T01:05:08.389745+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:05:08.389745+0000","last_undegraded":"2024-09-18T01:05:08.389745+0000","last_fullsized":"2024-09-18T01:05:08.389745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"122'578","reported_seq":5352,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389651+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:05:08.389651+0000","last_peered":"2024-09-18T01:05:08.389651+0000","last_clean":"2024-09-18T01:05:08.389651+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:05:08.389651+0000","last_undegraded":"2024-09-18T01:05:08.389651+0000","last_fullsized":"2024-09-18T01:05:08.389651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389731+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:05:08.389731+0000","last_peered":"2024-09-18T01:05:08.389731+0000","last_clean":"2024-09-18T01:05:08.389731+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:05:08.389731+0000","last_undegraded":"2024-09-18T01:05:08.389731+0000","last_fullsized":"2024-09-18T01:05:08.389731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7163,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395949+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:05:09.395949+0000","last_peered":"2024-09-18T01:05:09.395949+0000","last_clean":"2024-09-18T01:05:09.395949+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:05:09.395949+0000","last_undegraded":"2024-09-18T01:05:09.395949+0000","last_fullsized":"2024-09-18T01:05:09.395949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396147+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:05:09.396147+0000","last_peered":"2024-09-18T01:05:09.396147+0000","last_clean":"2024-09-18T01:05:09.396147+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:05:09.396147+0000","last_undegraded":"2024-09-18T01:05:09.396147+0000","last_fullsized":"2024-09-18T01:05:09.396147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10942,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929355+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:05:09.929355+0000","last_peered":"2024-09-18T01:05:09.929355+0000","last_clean":"2024-09-18T01:05:09.929355+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:05:09.929355+0000","last_undegraded":"2024-09-18T01:05:09.929355+0000","last_fullsized":"2024-09-18T01:05:09.929355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389817+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:05:08.389817+0000","last_peered":"2024-09-18T01:05:08.389817+0000","last_clean":"2024-09-18T01:05:08.389817+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:05:08.389817+0000","last_undegraded":"2024-09-18T01:05:08.389817+0000","last_fullsized":"2024-09-18T01:05:08.389817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6372,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396101+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:05:09.396101+0000","last_peered":"2024-09-18T01:05:09.396101+0000","last_clean":"2024-09-18T01:05:09.396101+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:05:09.396101+0000","last_undegraded":"2024-09-18T01:05:09.396101+0000","last_fullsized":"2024-09-18T01:05:09.396101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396210+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:05:09.396210+0000","last_peered":"2024-09-18T01:05:09.396210+0000","last_clean":"2024-09-18T01:05:09.396210+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:05:09.396210+0000","last_undegraded":"2024-09-18T01:05:09.396210+0000","last_fullsized":"2024-09-18T01:05:09.396210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805037+0000","last_change":"2024-09-18T01:04:56.307373+0000","last_active":"2024-09-18T01:05:09.805037+0000","last_peered":"2024-09-18T01:05:09.805037+0000","last_clean":"2024-09-18T01:05:09.805037+0000","last_became_active":"2024-09-18T01:04:56.305312+0000","last_became_peered":"2024-09-18T01:04:56.305312+0000","last_unstale":"2024-09-18T01:05:09.805037+0000","last_undegraded":"2024-09-18T01:05:09.805037+0000","last_fullsized":"2024-09-18T01:05:09.805037+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:50.866986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"125'1081","reported_seq":7538,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397772+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:05:09.397772+0000","last_peered":"2024-09-18T01:05:09.397772+0000","last_clean":"2024-09-18T01:05:09.397772+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:05:09.397772+0000","last_undegraded":"2024-09-18T01:05:09.397772+0000","last_fullsized":"2024-09-18T01:05:09.397772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804885+0000","last_change":"2024-09-18T01:04:53.308070+0000","last_active":"2024-09-18T01:05:09.804885+0000","last_peered":"2024-09-18T01:05:09.804885+0000","last_clean":"2024-09-18T01:05:09.804885+0000","last_became_active":"2024-09-18T01:04:53.302064+0000","last_became_peered":"2024-09-18T01:04:53.302064+0000","last_unstale":"2024-09-18T01:05:09.804885+0000","last_undegraded":"2024-09-18T01:05:09.804885+0000","last_fullsized":"2024-09-18T01:05:09.804885+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:39.560060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6751,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395474+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:05:09.395474+0000","last_peered":"2024-09-18T01:05:09.395474+0000","last_clean":"2024-09-18T01:05:09.395474+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:05:09.395474+0000","last_undegraded":"2024-09-18T01:05:09.395474+0000","last_fullsized":"2024-09-18T01:05:09.395474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929251+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:05:09.929251+0000","last_peered":"2024-09-18T01:05:09.929251+0000","last_clean":"2024-09-18T01:05:09.929251+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:05:09.929251+0000","last_undegraded":"2024-09-18T01:05:09.929251+0000","last_fullsized":"2024-09-18T01:05:09.929251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4599,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395962+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:05:09.395962+0000","last_peered":"2024-09-18T01:05:09.395962+0000","last_clean":"2024-09-18T01:05:09.395962+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:05:09.395962+0000","last_undegraded":"2024-09-18T01:05:09.395962+0000","last_fullsized":"2024-09-18T01:05:09.395962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389506+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:05:08.389506+0000","last_peered":"2024-09-18T01:05:08.389506+0000","last_clean":"2024-09-18T01:05:08.389506+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:05:08.389506+0000","last_undegraded":"2024-09-18T01:05:08.389506+0000","last_fullsized":"2024-09-18T01:05:08.389506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7250,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395678+0000","last_change":"2024-09-18T01:04:57.305044+0000","last_active":"2024-09-18T01:05:09.395678+0000","last_peered":"2024-09-18T01:05:09.395678+0000","last_clean":"2024-09-18T01:05:09.395678+0000","last_became_active":"2024-09-18T01:04:57.303668+0000","last_became_peered":"2024-09-18T01:04:57.303668+0000","last_unstale":"2024-09-18T01:05:09.395678+0000","last_undegraded":"2024-09-18T01:05:09.395678+0000","last_fullsized":"2024-09-18T01:05:09.395678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:29.198158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804899+0000","last_change":"2024-09-18T01:04:53.301659+0000","last_active":"2024-09-18T01:05:09.804899+0000","last_peered":"2024-09-18T01:05:09.804899+0000","last_clean":"2024-09-18T01:05:09.804899+0000","last_became_active":"2024-09-18T01:04:53.298761+0000","last_became_peered":"2024-09-18T01:04:53.298761+0000","last_unstale":"2024-09-18T01:05:09.804899+0000","last_undegraded":"2024-09-18T01:05:09.804899+0000","last_fullsized":"2024-09-18T01:05:09.804899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:51.494813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":430,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389519+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:05:08.389519+0000","last_peered":"2024-09-18T01:05:08.389519+0000","last_clean":"2024-09-18T01:05:08.389519+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:05:08.389519+0000","last_undegraded":"2024-09-18T01:05:08.389519+0000","last_fullsized":"2024-09-18T01:05:08.389519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8401,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'444","reported_seq":5116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395593+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:05:09.395593+0000","last_peered":"2024-09-18T01:05:09.395593+0000","last_clean":"2024-09-18T01:05:09.395593+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:05:09.395593+0000","last_undegraded":"2024-09-18T01:05:09.395593+0000","last_fullsized":"2024-09-18T01:05:09.395593+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"123'766","reported_seq":5462,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389682+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:05:08.389682+0000","last_peered":"2024-09-18T01:05:08.389682+0000","last_clean":"2024-09-18T01:05:08.389682+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:05:08.389682+0000","last_undegraded":"2024-09-18T01:05:08.389682+0000","last_fullsized":"2024-09-18T01:05:08.389682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'511","reported_seq":6976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395961+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:05:09.395961+0000","last_peered":"2024-09-18T01:05:09.395961+0000","last_clean":"2024-09-18T01:05:09.395961+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:05:09.395961+0000","last_undegraded":"2024-09-18T01:05:09.395961+0000","last_fullsized":"2024-09-18T01:05:09.395961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"114'371","reported_seq":6081,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804974+0000","last_change":"2024-09-18T01:04:56.306988+0000","last_active":"2024-09-18T01:05:09.804974+0000","last_peered":"2024-09-18T01:05:09.804974+0000","last_clean":"2024-09-18T01:05:09.804974+0000","last_became_active":"2024-09-18T01:04:56.304292+0000","last_became_peered":"2024-09-18T01:04:56.304292+0000","last_unstale":"2024-09-18T01:05:09.804974+0000","last_undegraded":"2024-09-18T01:05:09.804974+0000","last_fullsized":"2024-09-18T01:05:09.804974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:54.467271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'530","reported_seq":6215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389683+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:05:08.389683+0000","last_peered":"2024-09-18T01:05:08.389683+0000","last_clean":"2024-09-18T01:05:08.389683+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:05:08.389683+0000","last_undegraded":"2024-09-18T01:05:08.389683+0000","last_fullsized":"2024-09-18T01:05:08.389683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'399","reported_seq":6889,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805010+0000","last_change":"2024-09-18T01:04:56.308511+0000","last_active":"2024-09-18T01:05:09.805010+0000","last_peered":"2024-09-18T01:05:09.805010+0000","last_clean":"2024-09-18T01:05:09.805010+0000","last_became_active":"2024-09-18T01:04:56.306823+0000","last_became_peered":"2024-09-18T01:04:56.306823+0000","last_unstale":"2024-09-18T01:05:09.805010+0000","last_undegraded":"2024-09-18T01:05:09.805010+0000","last_fullsized":"2024-09-18T01:05:09.805010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:11:50.012365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"119'614","reported_seq":13226,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805091+0000","last_change":"2024-09-18T01:04:56.305438+0000","last_active":"2024-09-18T01:05:09.805091+0000","last_peered":"2024-09-18T01:05:09.805091+0000","last_clean":"2024-09-18T01:05:09.805091+0000","last_became_active":"2024-09-18T01:04:56.303159+0000","last_became_peered":"2024-09-18T01:04:56.303159+0000","last_unstale":"2024-09-18T01:05:09.805091+0000","last_undegraded":"2024-09-18T01:05:09.805091+0000","last_fullsized":"2024-09-18T01:05:09.805091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:12.250901+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"122'439","reported_seq":4641,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804829+0000","last_change":"2024-09-18T01:04:56.307127+0000","last_active":"2024-09-18T01:05:09.804829+0000","last_peered":"2024-09-18T01:05:09.804829+0000","last_clean":"2024-09-18T01:05:09.804829+0000","last_became_active":"2024-09-18T01:04:56.303327+0000","last_became_peered":"2024-09-18T01:04:56.303327+0000","last_unstale":"2024-09-18T01:05:09.804829+0000","last_undegraded":"2024-09-18T01:05:09.804829+0000","last_fullsized":"2024-09-18T01:05:09.804829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:45.602917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"121'408","reported_seq":4531,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389521+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:05:08.389521+0000","last_peered":"2024-09-18T01:05:08.389521+0000","last_clean":"2024-09-18T01:05:08.389521+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:05:08.389521+0000","last_undegraded":"2024-09-18T01:05:08.389521+0000","last_fullsized":"2024-09-18T01:05:08.389521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'976","reported_seq":10504,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395722+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:05:09.395722+0000","last_peered":"2024-09-18T01:05:09.395722+0000","last_clean":"2024-09-18T01:05:09.395722+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:05:09.395722+0000","last_undegraded":"2024-09-18T01:05:09.395722+0000","last_fullsized":"2024-09-18T01:05:09.395722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"124'313","reported_seq":4114,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389563+0000","last_change":"2024-09-18T01:04:56.304156+0000","last_active":"2024-09-18T01:05:08.389563+0000","last_peered":"2024-09-18T01:05:08.389563+0000","last_clean":"2024-09-18T01:05:08.389563+0000","last_became_active":"2024-09-18T01:04:56.302746+0000","last_became_peered":"2024-09-18T01:04:56.302746+0000","last_unstale":"2024-09-18T01:05:08.389563+0000","last_undegraded":"2024-09-18T01:05:08.389563+0000","last_fullsized":"2024-09-18T01:05:08.389563+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:41:30.202654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265609,"num_read_kb":1200484,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640292,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1761280,"data_stored":725496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1814528,"data_stored":1315845,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640292,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77324,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294516,"statfs":{"total":96636764160,"available":96557584384,"internally_reserved":0,"allocated":1757184,"data_stored":725477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640294,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64868,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306972,"statfs":{"total":96636764160,"available":96570339328,"internally_reserved":0,"allocated":1777664,"data_stored":730485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46864,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324976,"statfs":{"total":96636764160,"available":96588775424,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:05:13.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:05:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:05:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-18T01:05:12.371328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265726,"num_read_kb":1200585,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":323956,"kb_used_data":10460,"kb_used_omap":9,"kb_used_meta":313334,"kb_avail":565907084,"statfs":{"total":579820584960,"available":579488854016,"internally_reserved":0,"allocated":10711040,"data_stored":5548584,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":320854688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001394"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10557,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396239+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:05:09.396239+0000","last_peered":"2024-09-18T01:05:09.396239+0000","last_clean":"2024-09-18T01:05:09.396239+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:05:09.396239+0000","last_undegraded":"2024-09-18T01:05:09.396239+0000","last_fullsized":"2024-09-18T01:05:09.396239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'1473","reported_seq":7752,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929303+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:05:09.929303+0000","last_peered":"2024-09-18T01:05:09.929303+0000","last_clean":"2024-09-18T01:05:09.929303+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:05:09.929303+0000","last_undegraded":"2024-09-18T01:05:09.929303+0000","last_fullsized":"2024-09-18T01:05:09.929303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"124'455","reported_seq":7452,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805024+0000","last_change":"2024-09-18T01:04:56.309151+0000","last_active":"2024-09-18T01:05:09.805024+0000","last_peered":"2024-09-18T01:05:09.805024+0000","last_clean":"2024-09-18T01:05:09.805024+0000","last_became_active":"2024-09-18T01:04:56.307394+0000","last_became_peered":"2024-09-18T01:04:56.307394+0000","last_unstale":"2024-09-18T01:05:09.805024+0000","last_undegraded":"2024-09-18T01:05:09.805024+0000","last_fullsized":"2024-09-18T01:05:09.805024+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:35.044114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020314901999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"125'869","reported_seq":12809,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804894+0000","last_change":"2024-09-18T01:04:57.305367+0000","last_active":"2024-09-18T01:05:09.804894+0000","last_peered":"2024-09-18T01:05:09.804894+0000","last_clean":"2024-09-18T01:05:09.804894+0000","last_became_active":"2024-09-18T01:04:57.303594+0000","last_became_peered":"2024-09-18T01:04:57.303594+0000","last_unstale":"2024-09-18T01:05:09.804894+0000","last_undegraded":"2024-09-18T01:05:09.804894+0000","last_fullsized":"2024-09-18T01:05:09.804894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:51.883218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15908,"num_read_kb":43612,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"124'637","reported_seq":4531,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395893+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:05:09.395893+0000","last_peered":"2024-09-18T01:05:09.395893+0000","last_clean":"2024-09-18T01:05:09.395893+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:05:09.395893+0000","last_undegraded":"2024-09-18T01:05:09.395893+0000","last_fullsized":"2024-09-18T01:05:09.395893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"125'240","reported_seq":6067,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397675+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:05:09.397675+0000","last_peered":"2024-09-18T01:05:09.397675+0000","last_clean":"2024-09-18T01:05:09.397675+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:05:09.397675+0000","last_undegraded":"2024-09-18T01:05:09.397675+0000","last_fullsized":"2024-09-18T01:05:09.397675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"125'659","reported_seq":8756,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395703+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:05:09.395703+0000","last_peered":"2024-09-18T01:05:09.395703+0000","last_clean":"2024-09-18T01:05:09.395703+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:05:09.395703+0000","last_undegraded":"2024-09-18T01:05:09.395703+0000","last_fullsized":"2024-09-18T01:05:09.395703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"124'740","reported_seq":8844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"124'903","reported_seq":8734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396311+0000","last_change":"2024-09-18T01:04:57.304386+0000","last_active":"2024-09-18T01:05:09.396311+0000","last_peered":"2024-09-18T01:05:09.396311+0000","last_clean":"2024-09-18T01:05:09.396311+0000","last_became_active":"2024-09-18T01:04:57.302866+0000","last_became_peered":"2024-09-18T01:04:57.302866+0000","last_unstale":"2024-09-18T01:05:09.396311+0000","last_undegraded":"2024-09-18T01:05:09.396311+0000","last_fullsized":"2024-09-18T01:05:09.396311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:17.405000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"125'364","reported_seq":3882,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805054+0000","last_change":"2024-09-18T01:04:56.305126+0000","last_active":"2024-09-18T01:05:09.805054+0000","last_peered":"2024-09-18T01:05:09.805054+0000","last_clean":"2024-09-18T01:05:09.805054+0000","last_became_active":"2024-09-18T01:04:56.303131+0000","last_became_peered":"2024-09-18T01:04:56.303131+0000","last_unstale":"2024-09-18T01:05:09.805054+0000","last_undegraded":"2024-09-18T01:05:09.805054+0000","last_fullsized":"2024-09-18T01:05:09.805054+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:10.930635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"124'589","reported_seq":9916,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389745+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:05:08.389745+0000","last_peered":"2024-09-18T01:05:08.389745+0000","last_clean":"2024-09-18T01:05:08.389745+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:05:08.389745+0000","last_undegraded":"2024-09-18T01:05:08.389745+0000","last_fullsized":"2024-09-18T01:05:08.389745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"122'578","reported_seq":5352,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389651+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:05:08.389651+0000","last_peered":"2024-09-18T01:05:08.389651+0000","last_clean":"2024-09-18T01:05:08.389651+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:05:08.389651+0000","last_undegraded":"2024-09-18T01:05:08.389651+0000","last_fullsized":"2024-09-18T01:05:08.389651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389731+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:05:08.389731+0000","last_peered":"2024-09-18T01:05:08.389731+0000","last_clean":"2024-09-18T01:05:08.389731+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:05:08.389731+0000","last_undegraded":"2024-09-18T01:05:08.389731+0000","last_fullsized":"2024-09-18T01:05:08.389731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7163,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395949+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:05:09.395949+0000","last_peered":"2024-09-18T01:05:09.395949+0000","last_clean":"2024-09-18T01:05:09.395949+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:05:09.395949+0000","last_undegraded":"2024-09-18T01:05:09.395949+0000","last_fullsized":"2024-09-18T01:05:09.395949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396147+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:05:09.396147+0000","last_peered":"2024-09-18T01:05:09.396147+0000","last_clean":"2024-09-18T01:05:09.396147+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:05:09.396147+0000","last_undegraded":"2024-09-18T01:05:09.396147+0000","last_fullsized":"2024-09-18T01:05:09.396147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10942,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929355+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:05:09.929355+0000","last_peered":"2024-09-18T01:05:09.929355+0000","last_clean":"2024-09-18T01:05:09.929355+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:05:09.929355+0000","last_undegraded":"2024-09-18T01:05:09.929355+0000","last_fullsized":"2024-09-18T01:05:09.929355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389817+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:05:08.389817+0000","last_peered":"2024-09-18T01:05:08.389817+0000","last_clean":"2024-09-18T01:05:08.389817+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:05:08.389817+0000","last_undegraded":"2024-09-18T01:05:08.389817+0000","last_fullsized":"2024-09-18T01:05:08.389817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6372,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396101+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:05:09.396101+0000","last_peered":"2024-09-18T01:05:09.396101+0000","last_clean":"2024-09-18T01:05:09.396101+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:05:09.396101+0000","last_undegraded":"2024-09-18T01:05:09.396101+0000","last_fullsized":"2024-09-18T01:05:09.396101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396210+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:05:09.396210+0000","last_peered":"2024-09-18T01:05:09.396210+0000","last_clean":"2024-09-18T01:05:09.396210+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:05:09.396210+0000","last_undegraded":"2024-09-18T01:05:09.396210+0000","last_fullsized":"2024-09-18T01:05:09.396210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805037+0000","last_change":"2024-09-18T01:04:56.307373+0000","last_active":"2024-09-18T01:05:09.805037+0000","last_peered":"2024-09-18T01:05:09.805037+0000","last_clean":"2024-09-18T01:05:09.805037+0000","last_became_active":"2024-09-18T01:04:56.305312+0000","last_became_peered":"2024-09-18T01:04:56.305312+0000","last_unstale":"2024-09-18T01:05:09.805037+0000","last_undegraded":"2024-09-18T01:05:09.805037+0000","last_fullsized":"2024-09-18T01:05:09.805037+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:50.866986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"125'1081","reported_seq":7538,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397772+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:05:09.397772+0000","last_peered":"2024-09-18T01:05:09.397772+0000","last_clean":"2024-09-18T01:05:09.397772+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:05:09.397772+0000","last_undegraded":"2024-09-18T01:05:09.397772+0000","last_fullsized":"2024-09-18T01:05:09.397772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804885+0000","last_change":"2024-09-18T01:04:53.308070+0000","last_active":"2024-09-18T01:05:09.804885+0000","last_peered":"2024-09-18T01:05:09.804885+0000","last_clean":"2024-09-18T01:05:09.804885+0000","last_became_active":"2024-09-18T01:04:53.302064+0000","last_became_peered":"2024-09-18T01:04:53.302064+0000","last_unstale":"2024-09-18T01:05:09.804885+0000","last_undegraded":"2024-09-18T01:05:09.804885+0000","last_fullsized":"2024-09-18T01:05:09.804885+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:39.560060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6751,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395474+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:05:09.395474+0000","last_peered":"2024-09-18T01:05:09.395474+0000","last_clean":"2024-09-18T01:05:09.395474+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:05:09.395474+0000","last_undegraded":"2024-09-18T01:05:09.395474+0000","last_fullsized":"2024-09-18T01:05:09.395474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929251+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:05:09.929251+0000","last_peered":"2024-09-18T01:05:09.929251+0000","last_clean":"2024-09-18T01:05:09.929251+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:05:09.929251+0000","last_undegraded":"2024-09-18T01:05:09.929251+0000","last_fullsized":"2024-09-18T01:05:09.929251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4599,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395962+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:05:09.395962+0000","last_peered":"2024-09-18T01:05:09.395962+0000","last_clean":"2024-09-18T01:05:09.395962+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:05:09.395962+0000","last_undegraded":"2024-09-18T01:05:09.395962+0000","last_fullsized":"2024-09-18T01:05:09.395962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389506+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:05:08.389506+0000","last_peered":"2024-09-18T01:05:08.389506+0000","last_clean":"2024-09-18T01:05:08.389506+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:05:08.389506+0000","last_undegraded":"2024-09-18T01:05:08.389506+0000","last_fullsized":"2024-09-18T01:05:08.389506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7250,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395678+0000","last_change":"2024-09-18T01:04:57.305044+0000","last_active":"2024-09-18T01:05:09.395678+0000","last_peered":"2024-09-18T01:05:09.395678+0000","last_clean":"2024-09-18T01:05:09.395678+0000","last_became_active":"2024-09-18T01:04:57.303668+0000","last_became_peered":"2024-09-18T01:04:57.303668+0000","last_unstale":"2024-09-18T01:05:09.395678+0000","last_undegraded":"2024-09-18T01:05:09.395678+0000","last_fullsized":"2024-09-18T01:05:09.395678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:29.198158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804899+0000","last_change":"2024-09-18T01:04:53.301659+0000","last_active":"2024-09-18T01:05:09.804899+0000","last_peered":"2024-09-18T01:05:09.804899+0000","last_clean":"2024-09-18T01:05:09.804899+0000","last_became_active":"2024-09-18T01:04:53.298761+0000","last_became_peered":"2024-09-18T01:04:53.298761+0000","last_unstale":"2024-09-18T01:05:09.804899+0000","last_undegraded":"2024-09-18T01:05:09.804899+0000","last_fullsized":"2024-09-18T01:05:09.804899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:51.494813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":430,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389519+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:05:08.389519+0000","last_peered":"2024-09-18T01:05:08.389519+0000","last_clean":"2024-09-18T01:05:08.389519+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:05:08.389519+0000","last_undegraded":"2024-09-18T01:05:08.389519+0000","last_fullsized":"2024-09-18T01:05:08.389519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8401,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'444","reported_seq":5116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395593+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:05:09.395593+0000","last_peered":"2024-09-18T01:05:09.395593+0000","last_clean":"2024-09-18T01:05:09.395593+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:05:09.395593+0000","last_undegraded":"2024-09-18T01:05:09.395593+0000","last_fullsized":"2024-09-18T01:05:09.395593+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"123'766","reported_seq":5462,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389682+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:05:08.389682+0000","last_peered":"2024-09-18T01:05:08.389682+0000","last_clean":"2024-09-18T01:05:08.389682+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:05:08.389682+0000","last_undegraded":"2024-09-18T01:05:08.389682+0000","last_fullsized":"2024-09-18T01:05:08.389682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'511","reported_seq":6976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395961+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:05:09.395961+0000","last_peered":"2024-09-18T01:05:09.395961+0000","last_clean":"2024-09-18T01:05:09.395961+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:05:09.395961+0000","last_undegraded":"2024-09-18T01:05:09.395961+0000","last_fullsized":"2024-09-18T01:05:09.395961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"114'371","reported_seq":6081,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804974+0000","last_change":"2024-09-18T01:04:56.306988+0000","last_active":"2024-09-18T01:05:09.804974+0000","last_peered":"2024-09-18T01:05:09.804974+0000","last_clean":"2024-09-18T01:05:09.804974+0000","last_became_active":"2024-09-18T01:04:56.304292+0000","last_became_peered":"2024-09-18T01:04:56.304292+0000","last_unstale":"2024-09-18T01:05:09.804974+0000","last_undegraded":"2024-09-18T01:05:09.804974+0000","last_fullsized":"2024-09-18T01:05:09.804974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:54.467271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'530","reported_seq":6215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389683+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:05:08.389683+0000","last_peered":"2024-09-18T01:05:08.389683+0000","last_clean":"2024-09-18T01:05:08.389683+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:05:08.389683+0000","last_undegraded":"2024-09-18T01:05:08.389683+0000","last_fullsized":"2024-09-18T01:05:08.389683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'399","reported_seq":6889,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805010+0000","last_change":"2024-09-18T01:04:56.308511+0000","last_active":"2024-09-18T01:05:09.805010+0000","last_peered":"2024-09-18T01:05:09.805010+0000","last_clean":"2024-09-18T01:05:09.805010+0000","last_became_active":"2024-09-18T01:04:56.306823+0000","last_became_peered":"2024-09-18T01:04:56.306823+0000","last_unstale":"2024-09-18T01:05:09.805010+0000","last_undegraded":"2024-09-18T01:05:09.805010+0000","last_fullsized":"2024-09-18T01:05:09.805010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:11:50.012365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"119'614","reported_seq":13226,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805091+0000","last_change":"2024-09-18T01:04:56.305438+0000","last_active":"2024-09-18T01:05:09.805091+0000","last_peered":"2024-09-18T01:05:09.805091+0000","last_clean":"2024-09-18T01:05:09.805091+0000","last_became_active":"2024-09-18T01:04:56.303159+0000","last_became_peered":"2024-09-18T01:04:56.303159+0000","last_unstale":"2024-09-18T01:05:09.805091+0000","last_undegraded":"2024-09-18T01:05:09.805091+0000","last_fullsized":"2024-09-18T01:05:09.805091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:12.250901+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"122'439","reported_seq":4641,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804829+0000","last_change":"2024-09-18T01:04:56.307127+0000","last_active":"2024-09-18T01:05:09.804829+0000","last_peered":"2024-09-18T01:05:09.804829+0000","last_clean":"2024-09-18T01:05:09.804829+0000","last_became_active":"2024-09-18T01:04:56.303327+0000","last_became_peered":"2024-09-18T01:04:56.303327+0000","last_unstale":"2024-09-18T01:05:09.804829+0000","last_undegraded":"2024-09-18T01:05:09.804829+0000","last_fullsized":"2024-09-18T01:05:09.804829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:45.602917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"121'408","reported_seq":4531,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389521+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:05:08.389521+0000","last_peered":"2024-09-18T01:05:08.389521+0000","last_clean":"2024-09-18T01:05:08.389521+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:05:08.389521+0000","last_undegraded":"2024-09-18T01:05:08.389521+0000","last_fullsized":"2024-09-18T01:05:08.389521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'976","reported_seq":10504,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395722+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:05:09.395722+0000","last_peered":"2024-09-18T01:05:09.395722+0000","last_clean":"2024-09-18T01:05:09.395722+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:05:09.395722+0000","last_undegraded":"2024-09-18T01:05:09.395722+0000","last_fullsized":"2024-09-18T01:05:09.395722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"124'313","reported_seq":4114,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389563+0000","last_change":"2024-09-18T01:04:56.304156+0000","last_active":"2024-09-18T01:05:08.389563+0000","last_peered":"2024-09-18T01:05:08.389563+0000","last_clean":"2024-09-18T01:05:08.389563+0000","last_became_active":"2024-09-18T01:04:56.302746+0000","last_became_peered":"2024-09-18T01:04:56.302746+0000","last_unstale":"2024-09-18T01:05:08.389563+0000","last_undegraded":"2024-09-18T01:05:08.389563+0000","last_fullsized":"2024-09-18T01:05:08.389563+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:41:30.202654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265609,"num_read_kb":1200484,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640292,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1761280,"data_stored":725496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1814528,"data_stored":1315845,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640292,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77324,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294516,"statfs":{"total":96636764160,"available":96557584384,"internally_reserved":0,"allocated":1757184,"data_stored":725477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640294,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64868,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306972,"statfs":{"total":96636764160,"available":96570339328,"internally_reserved":0,"allocated":1777664,"data_stored":730485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46864,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324976,"statfs":{"total":96636764160,"available":96588775424,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:05:14.137 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T01:05:14.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-18T01:05:14.442 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T01:05:14.452 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T01:05:14.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:05:14.740 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:14.741 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:05:14.752 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-18T01:05:14.371721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265726,"num_read_kb":1200585,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":323956,"kb_used_data":10460,"kb_used_omap":9,"kb_used_meta":313334,"kb_avail":565907084,"statfs":{"total":579820584960,"available":579488854016,"internally_reserved":0,"allocated":10711040,"data_stored":5548584,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":320854688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.997780"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10557,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396239+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:05:09.396239+0000","last_peered":"2024-09-18T01:05:09.396239+0000","last_clean":"2024-09-18T01:05:09.396239+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:05:09.396239+0000","last_undegraded":"2024-09-18T01:05:09.396239+0000","last_fullsized":"2024-09-18T01:05:09.396239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'1473","reported_seq":7752,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929303+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:05:09.929303+0000","last_peered":"2024-09-18T01:05:09.929303+0000","last_clean":"2024-09-18T01:05:09.929303+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:05:09.929303+0000","last_undegraded":"2024-09-18T01:05:09.929303+0000","last_fullsized":"2024-09-18T01:05:09.929303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"124'455","reported_seq":7452,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805024+0000","last_change":"2024-09-18T01:04:56.309151+0000","last_active":"2024-09-18T01:05:09.805024+0000","last_peered":"2024-09-18T01:05:09.805024+0000","last_clean":"2024-09-18T01:05:09.805024+0000","last_became_active":"2024-09-18T01:04:56.307394+0000","last_became_peered":"2024-09-18T01:04:56.307394+0000","last_unstale":"2024-09-18T01:05:09.805024+0000","last_undegraded":"2024-09-18T01:05:09.805024+0000","last_fullsized":"2024-09-18T01:05:09.805024+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:35.044114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020314901999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"125'869","reported_seq":12809,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804894+0000","last_change":"2024-09-18T01:04:57.305367+0000","last_active":"2024-09-18T01:05:09.804894+0000","last_peered":"2024-09-18T01:05:09.804894+0000","last_clean":"2024-09-18T01:05:09.804894+0000","last_became_active":"2024-09-18T01:04:57.303594+0000","last_became_peered":"2024-09-18T01:04:57.303594+0000","last_unstale":"2024-09-18T01:05:09.804894+0000","last_undegraded":"2024-09-18T01:05:09.804894+0000","last_fullsized":"2024-09-18T01:05:09.804894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:51.883218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15908,"num_read_kb":43612,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"124'637","reported_seq":4531,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395893+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:05:09.395893+0000","last_peered":"2024-09-18T01:05:09.395893+0000","last_clean":"2024-09-18T01:05:09.395893+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:05:09.395893+0000","last_undegraded":"2024-09-18T01:05:09.395893+0000","last_fullsized":"2024-09-18T01:05:09.395893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"125'240","reported_seq":6067,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397675+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:05:09.397675+0000","last_peered":"2024-09-18T01:05:09.397675+0000","last_clean":"2024-09-18T01:05:09.397675+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:05:09.397675+0000","last_undegraded":"2024-09-18T01:05:09.397675+0000","last_fullsized":"2024-09-18T01:05:09.397675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"125'659","reported_seq":8756,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395703+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:05:09.395703+0000","last_peered":"2024-09-18T01:05:09.395703+0000","last_clean":"2024-09-18T01:05:09.395703+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:05:09.395703+0000","last_undegraded":"2024-09-18T01:05:09.395703+0000","last_fullsized":"2024-09-18T01:05:09.395703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"124'740","reported_seq":8844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"124'903","reported_seq":8734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396311+0000","last_change":"2024-09-18T01:04:57.304386+0000","last_active":"2024-09-18T01:05:09.396311+0000","last_peered":"2024-09-18T01:05:09.396311+0000","last_clean":"2024-09-18T01:05:09.396311+0000","last_became_active":"2024-09-18T01:04:57.302866+0000","last_became_peered":"2024-09-18T01:04:57.302866+0000","last_unstale":"2024-09-18T01:05:09.396311+0000","last_undegraded":"2024-09-18T01:05:09.396311+0000","last_fullsized":"2024-09-18T01:05:09.396311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:17.405000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"125'364","reported_seq":3882,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805054+0000","last_change":"2024-09-18T01:04:56.305126+0000","last_active":"2024-09-18T01:05:09.805054+0000","last_peered":"2024-09-18T01:05:09.805054+0000","last_clean":"2024-09-18T01:05:09.805054+0000","last_became_active":"2024-09-18T01:04:56.303131+0000","last_became_peered":"2024-09-18T01:04:56.303131+0000","last_unstale":"2024-09-18T01:05:09.805054+0000","last_undegraded":"2024-09-18T01:05:09.805054+0000","last_fullsized":"2024-09-18T01:05:09.805054+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:10.930635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"124'589","reported_seq":9916,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389745+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:05:08.389745+0000","last_peered":"2024-09-18T01:05:08.389745+0000","last_clean":"2024-09-18T01:05:08.389745+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:05:08.389745+0000","last_undegraded":"2024-09-18T01:05:08.389745+0000","last_fullsized":"2024-09-18T01:05:08.389745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"122'578","reported_seq":5352,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389651+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:05:08.389651+0000","last_peered":"2024-09-18T01:05:08.389651+0000","last_clean":"2024-09-18T01:05:08.389651+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:05:08.389651+0000","last_undegraded":"2024-09-18T01:05:08.389651+0000","last_fullsized":"2024-09-18T01:05:08.389651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389731+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:05:08.389731+0000","last_peered":"2024-09-18T01:05:08.389731+0000","last_clean":"2024-09-18T01:05:08.389731+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:05:08.389731+0000","last_undegraded":"2024-09-18T01:05:08.389731+0000","last_fullsized":"2024-09-18T01:05:08.389731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7163,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395949+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:05:09.395949+0000","last_peered":"2024-09-18T01:05:09.395949+0000","last_clean":"2024-09-18T01:05:09.395949+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:05:09.395949+0000","last_undegraded":"2024-09-18T01:05:09.395949+0000","last_fullsized":"2024-09-18T01:05:09.395949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396147+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:05:09.396147+0000","last_peered":"2024-09-18T01:05:09.396147+0000","last_clean":"2024-09-18T01:05:09.396147+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:05:09.396147+0000","last_undegraded":"2024-09-18T01:05:09.396147+0000","last_fullsized":"2024-09-18T01:05:09.396147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10942,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929355+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:05:09.929355+0000","last_peered":"2024-09-18T01:05:09.929355+0000","last_clean":"2024-09-18T01:05:09.929355+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:05:09.929355+0000","last_undegraded":"2024-09-18T01:05:09.929355+0000","last_fullsized":"2024-09-18T01:05:09.929355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389817+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:05:08.389817+0000","last_peered":"2024-09-18T01:05:08.389817+0000","last_clean":"2024-09-18T01:05:08.389817+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:05:08.389817+0000","last_undegraded":"2024-09-18T01:05:08.389817+0000","last_fullsized":"2024-09-18T01:05:08.389817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6372,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396101+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:05:09.396101+0000","last_peered":"2024-09-18T01:05:09.396101+0000","last_clean":"2024-09-18T01:05:09.396101+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:05:09.396101+0000","last_undegraded":"2024-09-18T01:05:09.396101+0000","last_fullsized":"2024-09-18T01:05:09.396101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396210+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:05:09.396210+0000","last_peered":"2024-09-18T01:05:09.396210+0000","last_clean":"2024-09-18T01:05:09.396210+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:05:09.396210+0000","last_undegraded":"2024-09-18T01:05:09.396210+0000","last_fullsized":"2024-09-18T01:05:09.396210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805037+0000","last_change":"2024-09-18T01:04:56.307373+0000","last_active":"2024-09-18T01:05:09.805037+0000","last_peered":"2024-09-18T01:05:09.805037+0000","last_clean":"2024-09-18T01:05:09.805037+0000","last_became_active":"2024-09-18T01:04:56.305312+0000","last_became_peered":"2024-09-18T01:04:56.305312+0000","last_unstale":"2024-09-18T01:05:09.805037+0000","last_undegraded":"2024-09-18T01:05:09.805037+0000","last_fullsized":"2024-09-18T01:05:09.805037+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:50.866986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"125'1081","reported_seq":7538,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397772+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:05:09.397772+0000","last_peered":"2024-09-18T01:05:09.397772+0000","last_clean":"2024-09-18T01:05:09.397772+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:05:09.397772+0000","last_undegraded":"2024-09-18T01:05:09.397772+0000","last_fullsized":"2024-09-18T01:05:09.397772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804885+0000","last_change":"2024-09-18T01:04:53.308070+0000","last_active":"2024-09-18T01:05:09.804885+0000","last_peered":"2024-09-18T01:05:09.804885+0000","last_clean":"2024-09-18T01:05:09.804885+0000","last_became_active":"2024-09-18T01:04:53.302064+0000","last_became_peered":"2024-09-18T01:04:53.302064+0000","last_unstale":"2024-09-18T01:05:09.804885+0000","last_undegraded":"2024-09-18T01:05:09.804885+0000","last_fullsized":"2024-09-18T01:05:09.804885+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:39.560060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6751,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395474+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:05:09.395474+0000","last_peered":"2024-09-18T01:05:09.395474+0000","last_clean":"2024-09-18T01:05:09.395474+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:05:09.395474+0000","last_undegraded":"2024-09-18T01:05:09.395474+0000","last_fullsized":"2024-09-18T01:05:09.395474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929251+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:05:09.929251+0000","last_peered":"2024-09-18T01:05:09.929251+0000","last_clean":"2024-09-18T01:05:09.929251+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:05:09.929251+0000","last_undegraded":"2024-09-18T01:05:09.929251+0000","last_fullsized":"2024-09-18T01:05:09.929251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4599,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395962+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:05:09.395962+0000","last_peered":"2024-09-18T01:05:09.395962+0000","last_clean":"2024-09-18T01:05:09.395962+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:05:09.395962+0000","last_undegraded":"2024-09-18T01:05:09.395962+0000","last_fullsized":"2024-09-18T01:05:09.395962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389506+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:05:08.389506+0000","last_peered":"2024-09-18T01:05:08.389506+0000","last_clean":"2024-09-18T01:05:08.389506+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:05:08.389506+0000","last_undegraded":"2024-09-18T01:05:08.389506+0000","last_fullsized":"2024-09-18T01:05:08.389506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7250,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395678+0000","last_change":"2024-09-18T01:04:57.305044+0000","last_active":"2024-09-18T01:05:09.395678+0000","last_peered":"2024-09-18T01:05:09.395678+0000","last_clean":"2024-09-18T01:05:09.395678+0000","last_became_active":"2024-09-18T01:04:57.303668+0000","last_became_peered":"2024-09-18T01:04:57.303668+0000","last_unstale":"2024-09-18T01:05:09.395678+0000","last_undegraded":"2024-09-18T01:05:09.395678+0000","last_fullsized":"2024-09-18T01:05:09.395678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:29.198158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804899+0000","last_change":"2024-09-18T01:04:53.301659+0000","last_active":"2024-09-18T01:05:09.804899+0000","last_peered":"2024-09-18T01:05:09.804899+0000","last_clean":"2024-09-18T01:05:09.804899+0000","last_became_active":"2024-09-18T01:04:53.298761+0000","last_became_peered":"2024-09-18T01:04:53.298761+0000","last_unstale":"2024-09-18T01:05:09.804899+0000","last_undegraded":"2024-09-18T01:05:09.804899+0000","last_fullsized":"2024-09-18T01:05:09.804899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:51.494813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":430,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389519+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:05:08.389519+0000","last_peered":"2024-09-18T01:05:08.389519+0000","last_clean":"2024-09-18T01:05:08.389519+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:05:08.389519+0000","last_undegraded":"2024-09-18T01:05:08.389519+0000","last_fullsized":"2024-09-18T01:05:08.389519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8401,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'444","reported_seq":5116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395593+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:05:09.395593+0000","last_peered":"2024-09-18T01:05:09.395593+0000","last_clean":"2024-09-18T01:05:09.395593+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:05:09.395593+0000","last_undegraded":"2024-09-18T01:05:09.395593+0000","last_fullsized":"2024-09-18T01:05:09.395593+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"123'766","reported_seq":5462,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389682+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:05:08.389682+0000","last_peered":"2024-09-18T01:05:08.389682+0000","last_clean":"2024-09-18T01:05:08.389682+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:05:08.389682+0000","last_undegraded":"2024-09-18T01:05:08.389682+0000","last_fullsized":"2024-09-18T01:05:08.389682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'511","reported_seq":6976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395961+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:05:09.395961+0000","last_peered":"2024-09-18T01:05:09.395961+0000","last_clean":"2024-09-18T01:05:09.395961+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:05:09.395961+0000","last_undegraded":"2024-09-18T01:05:09.395961+0000","last_fullsized":"2024-09-18T01:05:09.395961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"114'371","reported_seq":6081,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804974+0000","last_change":"2024-09-18T01:04:56.306988+0000","last_active":"2024-09-18T01:05:09.804974+0000","last_peered":"2024-09-18T01:05:09.804974+0000","last_clean":"2024-09-18T01:05:09.804974+0000","last_became_active":"2024-09-18T01:04:56.304292+0000","last_became_peered":"2024-09-18T01:04:56.304292+0000","last_unstale":"2024-09-18T01:05:09.804974+0000","last_undegraded":"2024-09-18T01:05:09.804974+0000","last_fullsized":"2024-09-18T01:05:09.804974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:54.467271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'530","reported_seq":6215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389683+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:05:08.389683+0000","last_peered":"2024-09-18T01:05:08.389683+0000","last_clean":"2024-09-18T01:05:08.389683+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:05:08.389683+0000","last_undegraded":"2024-09-18T01:05:08.389683+0000","last_fullsized":"2024-09-18T01:05:08.389683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'399","reported_seq":6889,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805010+0000","last_change":"2024-09-18T01:04:56.308511+0000","last_active":"2024-09-18T01:05:09.805010+0000","last_peered":"2024-09-18T01:05:09.805010+0000","last_clean":"2024-09-18T01:05:09.805010+0000","last_became_active":"2024-09-18T01:04:56.306823+0000","last_became_peered":"2024-09-18T01:04:56.306823+0000","last_unstale":"2024-09-18T01:05:09.805010+0000","last_undegraded":"2024-09-18T01:05:09.805010+0000","last_fullsized":"2024-09-18T01:05:09.805010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:11:50.012365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"119'614","reported_seq":13226,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805091+0000","last_change":"2024-09-18T01:04:56.305438+0000","last_active":"2024-09-18T01:05:09.805091+0000","last_peered":"2024-09-18T01:05:09.805091+0000","last_clean":"2024-09-18T01:05:09.805091+0000","last_became_active":"2024-09-18T01:04:56.303159+0000","last_became_peered":"2024-09-18T01:04:56.303159+0000","last_unstale":"2024-09-18T01:05:09.805091+0000","last_undegraded":"2024-09-18T01:05:09.805091+0000","last_fullsized":"2024-09-18T01:05:09.805091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:12.250901+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"122'439","reported_seq":4641,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804829+0000","last_change":"2024-09-18T01:04:56.307127+0000","last_active":"2024-09-18T01:05:09.804829+0000","last_peered":"2024-09-18T01:05:09.804829+0000","last_clean":"2024-09-18T01:05:09.804829+0000","last_became_active":"2024-09-18T01:04:56.303327+0000","last_became_peered":"2024-09-18T01:04:56.303327+0000","last_unstale":"2024-09-18T01:05:09.804829+0000","last_undegraded":"2024-09-18T01:05:09.804829+0000","last_fullsized":"2024-09-18T01:05:09.804829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:45.602917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"121'408","reported_seq":4531,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389521+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:05:08.389521+0000","last_peered":"2024-09-18T01:05:08.389521+0000","last_clean":"2024-09-18T01:05:08.389521+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:05:08.389521+0000","last_undegraded":"2024-09-18T01:05:08.389521+0000","last_fullsized":"2024-09-18T01:05:08.389521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'976","reported_seq":10504,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395722+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:05:09.395722+0000","last_peered":"2024-09-18T01:05:09.395722+0000","last_clean":"2024-09-18T01:05:09.395722+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:05:09.395722+0000","last_undegraded":"2024-09-18T01:05:09.395722+0000","last_fullsized":"2024-09-18T01:05:09.395722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"124'313","reported_seq":4114,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389563+0000","last_change":"2024-09-18T01:04:56.304156+0000","last_active":"2024-09-18T01:05:08.389563+0000","last_peered":"2024-09-18T01:05:08.389563+0000","last_clean":"2024-09-18T01:05:08.389563+0000","last_became_active":"2024-09-18T01:04:56.302746+0000","last_became_peered":"2024-09-18T01:04:56.302746+0000","last_unstale":"2024-09-18T01:05:08.389563+0000","last_undegraded":"2024-09-18T01:05:08.389563+0000","last_fullsized":"2024-09-18T01:05:08.389563+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:41:30.202654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265609,"num_read_kb":1200484,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640292,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1761280,"data_stored":725496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1814528,"data_stored":1315845,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640292,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77324,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294516,"statfs":{"total":96636764160,"available":96557584384,"internally_reserved":0,"allocated":1757184,"data_stored":725477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640294,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64868,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306972,"statfs":{"total":96636764160,"available":96570339328,"internally_reserved":0,"allocated":1777664,"data_stored":730485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46864,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324976,"statfs":{"total":96636764160,"available":96588775424,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:05:14.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:05:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:15.044 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:05:15.056 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-18T01:05:14.371721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265726,"num_read_kb":1200585,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":323956,"kb_used_data":10460,"kb_used_omap":9,"kb_used_meta":313334,"kb_avail":565907084,"statfs":{"total":579820584960,"available":579488854016,"internally_reserved":0,"allocated":10711040,"data_stored":5548584,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":320854688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.997780"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10557,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396239+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:05:09.396239+0000","last_peered":"2024-09-18T01:05:09.396239+0000","last_clean":"2024-09-18T01:05:09.396239+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:05:09.396239+0000","last_undegraded":"2024-09-18T01:05:09.396239+0000","last_fullsized":"2024-09-18T01:05:09.396239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'1473","reported_seq":7752,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929303+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:05:09.929303+0000","last_peered":"2024-09-18T01:05:09.929303+0000","last_clean":"2024-09-18T01:05:09.929303+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:05:09.929303+0000","last_undegraded":"2024-09-18T01:05:09.929303+0000","last_fullsized":"2024-09-18T01:05:09.929303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"124'455","reported_seq":7452,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805024+0000","last_change":"2024-09-18T01:04:56.309151+0000","last_active":"2024-09-18T01:05:09.805024+0000","last_peered":"2024-09-18T01:05:09.805024+0000","last_clean":"2024-09-18T01:05:09.805024+0000","last_became_active":"2024-09-18T01:04:56.307394+0000","last_became_peered":"2024-09-18T01:04:56.307394+0000","last_unstale":"2024-09-18T01:05:09.805024+0000","last_undegraded":"2024-09-18T01:05:09.805024+0000","last_fullsized":"2024-09-18T01:05:09.805024+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:35.044114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020314901999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"125'869","reported_seq":12809,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804894+0000","last_change":"2024-09-18T01:04:57.305367+0000","last_active":"2024-09-18T01:05:09.804894+0000","last_peered":"2024-09-18T01:05:09.804894+0000","last_clean":"2024-09-18T01:05:09.804894+0000","last_became_active":"2024-09-18T01:04:57.303594+0000","last_became_peered":"2024-09-18T01:04:57.303594+0000","last_unstale":"2024-09-18T01:05:09.804894+0000","last_undegraded":"2024-09-18T01:05:09.804894+0000","last_fullsized":"2024-09-18T01:05:09.804894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:51.883218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15908,"num_read_kb":43612,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"124'637","reported_seq":4531,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395893+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:05:09.395893+0000","last_peered":"2024-09-18T01:05:09.395893+0000","last_clean":"2024-09-18T01:05:09.395893+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:05:09.395893+0000","last_undegraded":"2024-09-18T01:05:09.395893+0000","last_fullsized":"2024-09-18T01:05:09.395893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"125'240","reported_seq":6067,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397675+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:05:09.397675+0000","last_peered":"2024-09-18T01:05:09.397675+0000","last_clean":"2024-09-18T01:05:09.397675+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:05:09.397675+0000","last_undegraded":"2024-09-18T01:05:09.397675+0000","last_fullsized":"2024-09-18T01:05:09.397675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"125'659","reported_seq":8756,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395703+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:05:09.395703+0000","last_peered":"2024-09-18T01:05:09.395703+0000","last_clean":"2024-09-18T01:05:09.395703+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:05:09.395703+0000","last_undegraded":"2024-09-18T01:05:09.395703+0000","last_fullsized":"2024-09-18T01:05:09.395703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"124'740","reported_seq":8844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"124'903","reported_seq":8734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396311+0000","last_change":"2024-09-18T01:04:57.304386+0000","last_active":"2024-09-18T01:05:09.396311+0000","last_peered":"2024-09-18T01:05:09.396311+0000","last_clean":"2024-09-18T01:05:09.396311+0000","last_became_active":"2024-09-18T01:04:57.302866+0000","last_became_peered":"2024-09-18T01:04:57.302866+0000","last_unstale":"2024-09-18T01:05:09.396311+0000","last_undegraded":"2024-09-18T01:05:09.396311+0000","last_fullsized":"2024-09-18T01:05:09.396311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:17.405000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"125'364","reported_seq":3882,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805054+0000","last_change":"2024-09-18T01:04:56.305126+0000","last_active":"2024-09-18T01:05:09.805054+0000","last_peered":"2024-09-18T01:05:09.805054+0000","last_clean":"2024-09-18T01:05:09.805054+0000","last_became_active":"2024-09-18T01:04:56.303131+0000","last_became_peered":"2024-09-18T01:04:56.303131+0000","last_unstale":"2024-09-18T01:05:09.805054+0000","last_undegraded":"2024-09-18T01:05:09.805054+0000","last_fullsized":"2024-09-18T01:05:09.805054+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:10.930635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"124'589","reported_seq":9916,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389745+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:05:08.389745+0000","last_peered":"2024-09-18T01:05:08.389745+0000","last_clean":"2024-09-18T01:05:08.389745+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:05:08.389745+0000","last_undegraded":"2024-09-18T01:05:08.389745+0000","last_fullsized":"2024-09-18T01:05:08.389745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"122'578","reported_seq":5352,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389651+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:05:08.389651+0000","last_peered":"2024-09-18T01:05:08.389651+0000","last_clean":"2024-09-18T01:05:08.389651+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:05:08.389651+0000","last_undegraded":"2024-09-18T01:05:08.389651+0000","last_fullsized":"2024-09-18T01:05:08.389651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389731+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:05:08.389731+0000","last_peered":"2024-09-18T01:05:08.389731+0000","last_clean":"2024-09-18T01:05:08.389731+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:05:08.389731+0000","last_undegraded":"2024-09-18T01:05:08.389731+0000","last_fullsized":"2024-09-18T01:05:08.389731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7163,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395949+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:05:09.395949+0000","last_peered":"2024-09-18T01:05:09.395949+0000","last_clean":"2024-09-18T01:05:09.395949+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:05:09.395949+0000","last_undegraded":"2024-09-18T01:05:09.395949+0000","last_fullsized":"2024-09-18T01:05:09.395949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396147+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:05:09.396147+0000","last_peered":"2024-09-18T01:05:09.396147+0000","last_clean":"2024-09-18T01:05:09.396147+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:05:09.396147+0000","last_undegraded":"2024-09-18T01:05:09.396147+0000","last_fullsized":"2024-09-18T01:05:09.396147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10942,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929355+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:05:09.929355+0000","last_peered":"2024-09-18T01:05:09.929355+0000","last_clean":"2024-09-18T01:05:09.929355+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:05:09.929355+0000","last_undegraded":"2024-09-18T01:05:09.929355+0000","last_fullsized":"2024-09-18T01:05:09.929355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389817+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:05:08.389817+0000","last_peered":"2024-09-18T01:05:08.389817+0000","last_clean":"2024-09-18T01:05:08.389817+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:05:08.389817+0000","last_undegraded":"2024-09-18T01:05:08.389817+0000","last_fullsized":"2024-09-18T01:05:08.389817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6372,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396101+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:05:09.396101+0000","last_peered":"2024-09-18T01:05:09.396101+0000","last_clean":"2024-09-18T01:05:09.396101+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:05:09.396101+0000","last_undegraded":"2024-09-18T01:05:09.396101+0000","last_fullsized":"2024-09-18T01:05:09.396101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396210+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:05:09.396210+0000","last_peered":"2024-09-18T01:05:09.396210+0000","last_clean":"2024-09-18T01:05:09.396210+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:05:09.396210+0000","last_undegraded":"2024-09-18T01:05:09.396210+0000","last_fullsized":"2024-09-18T01:05:09.396210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805037+0000","last_change":"2024-09-18T01:04:56.307373+0000","last_active":"2024-09-18T01:05:09.805037+0000","last_peered":"2024-09-18T01:05:09.805037+0000","last_clean":"2024-09-18T01:05:09.805037+0000","last_became_active":"2024-09-18T01:04:56.305312+0000","last_became_peered":"2024-09-18T01:04:56.305312+0000","last_unstale":"2024-09-18T01:05:09.805037+0000","last_undegraded":"2024-09-18T01:05:09.805037+0000","last_fullsized":"2024-09-18T01:05:09.805037+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:50.866986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"125'1081","reported_seq":7538,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397772+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:05:09.397772+0000","last_peered":"2024-09-18T01:05:09.397772+0000","last_clean":"2024-09-18T01:05:09.397772+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:05:09.397772+0000","last_undegraded":"2024-09-18T01:05:09.397772+0000","last_fullsized":"2024-09-18T01:05:09.397772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804885+0000","last_change":"2024-09-18T01:04:53.308070+0000","last_active":"2024-09-18T01:05:09.804885+0000","last_peered":"2024-09-18T01:05:09.804885+0000","last_clean":"2024-09-18T01:05:09.804885+0000","last_became_active":"2024-09-18T01:04:53.302064+0000","last_became_peered":"2024-09-18T01:04:53.302064+0000","last_unstale":"2024-09-18T01:05:09.804885+0000","last_undegraded":"2024-09-18T01:05:09.804885+0000","last_fullsized":"2024-09-18T01:05:09.804885+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:39.560060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6751,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395474+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:05:09.395474+0000","last_peered":"2024-09-18T01:05:09.395474+0000","last_clean":"2024-09-18T01:05:09.395474+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:05:09.395474+0000","last_undegraded":"2024-09-18T01:05:09.395474+0000","last_fullsized":"2024-09-18T01:05:09.395474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929251+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:05:09.929251+0000","last_peered":"2024-09-18T01:05:09.929251+0000","last_clean":"2024-09-18T01:05:09.929251+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:05:09.929251+0000","last_undegraded":"2024-09-18T01:05:09.929251+0000","last_fullsized":"2024-09-18T01:05:09.929251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4599,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395962+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:05:09.395962+0000","last_peered":"2024-09-18T01:05:09.395962+0000","last_clean":"2024-09-18T01:05:09.395962+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:05:09.395962+0000","last_undegraded":"2024-09-18T01:05:09.395962+0000","last_fullsized":"2024-09-18T01:05:09.395962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389506+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:05:08.389506+0000","last_peered":"2024-09-18T01:05:08.389506+0000","last_clean":"2024-09-18T01:05:08.389506+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:05:08.389506+0000","last_undegraded":"2024-09-18T01:05:08.389506+0000","last_fullsized":"2024-09-18T01:05:08.389506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7250,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395678+0000","last_change":"2024-09-18T01:04:57.305044+0000","last_active":"2024-09-18T01:05:09.395678+0000","last_peered":"2024-09-18T01:05:09.395678+0000","last_clean":"2024-09-18T01:05:09.395678+0000","last_became_active":"2024-09-18T01:04:57.303668+0000","last_became_peered":"2024-09-18T01:04:57.303668+0000","last_unstale":"2024-09-18T01:05:09.395678+0000","last_undegraded":"2024-09-18T01:05:09.395678+0000","last_fullsized":"2024-09-18T01:05:09.395678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:29.198158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804899+0000","last_change":"2024-09-18T01:04:53.301659+0000","last_active":"2024-09-18T01:05:09.804899+0000","last_peered":"2024-09-18T01:05:09.804899+0000","last_clean":"2024-09-18T01:05:09.804899+0000","last_became_active":"2024-09-18T01:04:53.298761+0000","last_became_peered":"2024-09-18T01:04:53.298761+0000","last_unstale":"2024-09-18T01:05:09.804899+0000","last_undegraded":"2024-09-18T01:05:09.804899+0000","last_fullsized":"2024-09-18T01:05:09.804899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:51.494813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":430,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389519+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:05:08.389519+0000","last_peered":"2024-09-18T01:05:08.389519+0000","last_clean":"2024-09-18T01:05:08.389519+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:05:08.389519+0000","last_undegraded":"2024-09-18T01:05:08.389519+0000","last_fullsized":"2024-09-18T01:05:08.389519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8401,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'444","reported_seq":5116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395593+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:05:09.395593+0000","last_peered":"2024-09-18T01:05:09.395593+0000","last_clean":"2024-09-18T01:05:09.395593+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:05:09.395593+0000","last_undegraded":"2024-09-18T01:05:09.395593+0000","last_fullsized":"2024-09-18T01:05:09.395593+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"123'766","reported_seq":5462,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389682+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:05:08.389682+0000","last_peered":"2024-09-18T01:05:08.389682+0000","last_clean":"2024-09-18T01:05:08.389682+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:05:08.389682+0000","last_undegraded":"2024-09-18T01:05:08.389682+0000","last_fullsized":"2024-09-18T01:05:08.389682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'511","reported_seq":6976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395961+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:05:09.395961+0000","last_peered":"2024-09-18T01:05:09.395961+0000","last_clean":"2024-09-18T01:05:09.395961+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:05:09.395961+0000","last_undegraded":"2024-09-18T01:05:09.395961+0000","last_fullsized":"2024-09-18T01:05:09.395961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"114'371","reported_seq":6081,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804974+0000","last_change":"2024-09-18T01:04:56.306988+0000","last_active":"2024-09-18T01:05:09.804974+0000","last_peered":"2024-09-18T01:05:09.804974+0000","last_clean":"2024-09-18T01:05:09.804974+0000","last_became_active":"2024-09-18T01:04:56.304292+0000","last_became_peered":"2024-09-18T01:04:56.304292+0000","last_unstale":"2024-09-18T01:05:09.804974+0000","last_undegraded":"2024-09-18T01:05:09.804974+0000","last_fullsized":"2024-09-18T01:05:09.804974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:54.467271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'530","reported_seq":6215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389683+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:05:08.389683+0000","last_peered":"2024-09-18T01:05:08.389683+0000","last_clean":"2024-09-18T01:05:08.389683+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:05:08.389683+0000","last_undegraded":"2024-09-18T01:05:08.389683+0000","last_fullsized":"2024-09-18T01:05:08.389683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'399","reported_seq":6889,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805010+0000","last_change":"2024-09-18T01:04:56.308511+0000","last_active":"2024-09-18T01:05:09.805010+0000","last_peered":"2024-09-18T01:05:09.805010+0000","last_clean":"2024-09-18T01:05:09.805010+0000","last_became_active":"2024-09-18T01:04:56.306823+0000","last_became_peered":"2024-09-18T01:04:56.306823+0000","last_unstale":"2024-09-18T01:05:09.805010+0000","last_undegraded":"2024-09-18T01:05:09.805010+0000","last_fullsized":"2024-09-18T01:05:09.805010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:11:50.012365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"119'614","reported_seq":13226,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805091+0000","last_change":"2024-09-18T01:04:56.305438+0000","last_active":"2024-09-18T01:05:09.805091+0000","last_peered":"2024-09-18T01:05:09.805091+0000","last_clean":"2024-09-18T01:05:09.805091+0000","last_became_active":"2024-09-18T01:04:56.303159+0000","last_became_peered":"2024-09-18T01:04:56.303159+0000","last_unstale":"2024-09-18T01:05:09.805091+0000","last_undegraded":"2024-09-18T01:05:09.805091+0000","last_fullsized":"2024-09-18T01:05:09.805091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:12.250901+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"122'439","reported_seq":4641,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804829+0000","last_change":"2024-09-18T01:04:56.307127+0000","last_active":"2024-09-18T01:05:09.804829+0000","last_peered":"2024-09-18T01:05:09.804829+0000","last_clean":"2024-09-18T01:05:09.804829+0000","last_became_active":"2024-09-18T01:04:56.303327+0000","last_became_peered":"2024-09-18T01:04:56.303327+0000","last_unstale":"2024-09-18T01:05:09.804829+0000","last_undegraded":"2024-09-18T01:05:09.804829+0000","last_fullsized":"2024-09-18T01:05:09.804829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:45.602917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"121'408","reported_seq":4531,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389521+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:05:08.389521+0000","last_peered":"2024-09-18T01:05:08.389521+0000","last_clean":"2024-09-18T01:05:08.389521+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:05:08.389521+0000","last_undegraded":"2024-09-18T01:05:08.389521+0000","last_fullsized":"2024-09-18T01:05:08.389521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'976","reported_seq":10504,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395722+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:05:09.395722+0000","last_peered":"2024-09-18T01:05:09.395722+0000","last_clean":"2024-09-18T01:05:09.395722+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:05:09.395722+0000","last_undegraded":"2024-09-18T01:05:09.395722+0000","last_fullsized":"2024-09-18T01:05:09.395722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"124'313","reported_seq":4114,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389563+0000","last_change":"2024-09-18T01:04:56.304156+0000","last_active":"2024-09-18T01:05:08.389563+0000","last_peered":"2024-09-18T01:05:08.389563+0000","last_clean":"2024-09-18T01:05:08.389563+0000","last_became_active":"2024-09-18T01:04:56.302746+0000","last_became_peered":"2024-09-18T01:04:56.302746+0000","last_unstale":"2024-09-18T01:05:08.389563+0000","last_undegraded":"2024-09-18T01:05:08.389563+0000","last_fullsized":"2024-09-18T01:05:08.389563+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:41:30.202654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265609,"num_read_kb":1200484,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640292,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1761280,"data_stored":725496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1814528,"data_stored":1315845,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640292,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77324,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294516,"statfs":{"total":96636764160,"available":96557584384,"internally_reserved":0,"allocated":1757184,"data_stored":725477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640294,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64868,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306972,"statfs":{"total":96636764160,"available":96570339328,"internally_reserved":0,"allocated":1777664,"data_stored":730485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46864,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324976,"statfs":{"total":96636764160,"available":96588775424,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:05:15.058 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T01:05:15.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:05:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:15.349 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:05:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-18T01:05:14.371721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265726,"num_read_kb":1200585,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":323956,"kb_used_data":10460,"kb_used_omap":9,"kb_used_meta":313334,"kb_avail":565907084,"statfs":{"total":579820584960,"available":579488854016,"internally_reserved":0,"allocated":10711040,"data_stored":5548584,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":320854688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.997780"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10557,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396239+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:05:09.396239+0000","last_peered":"2024-09-18T01:05:09.396239+0000","last_clean":"2024-09-18T01:05:09.396239+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:05:09.396239+0000","last_undegraded":"2024-09-18T01:05:09.396239+0000","last_fullsized":"2024-09-18T01:05:09.396239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'1473","reported_seq":7752,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929303+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:05:09.929303+0000","last_peered":"2024-09-18T01:05:09.929303+0000","last_clean":"2024-09-18T01:05:09.929303+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:05:09.929303+0000","last_undegraded":"2024-09-18T01:05:09.929303+0000","last_fullsized":"2024-09-18T01:05:09.929303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"124'455","reported_seq":7452,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805024+0000","last_change":"2024-09-18T01:04:56.309151+0000","last_active":"2024-09-18T01:05:09.805024+0000","last_peered":"2024-09-18T01:05:09.805024+0000","last_clean":"2024-09-18T01:05:09.805024+0000","last_became_active":"2024-09-18T01:04:56.307394+0000","last_became_peered":"2024-09-18T01:04:56.307394+0000","last_unstale":"2024-09-18T01:05:09.805024+0000","last_undegraded":"2024-09-18T01:05:09.805024+0000","last_fullsized":"2024-09-18T01:05:09.805024+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:35.044114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020314901999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"125'869","reported_seq":12809,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804894+0000","last_change":"2024-09-18T01:04:57.305367+0000","last_active":"2024-09-18T01:05:09.804894+0000","last_peered":"2024-09-18T01:05:09.804894+0000","last_clean":"2024-09-18T01:05:09.804894+0000","last_became_active":"2024-09-18T01:04:57.303594+0000","last_became_peered":"2024-09-18T01:04:57.303594+0000","last_unstale":"2024-09-18T01:05:09.804894+0000","last_undegraded":"2024-09-18T01:05:09.804894+0000","last_fullsized":"2024-09-18T01:05:09.804894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:51.883218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15908,"num_read_kb":43612,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"124'637","reported_seq":4531,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395893+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:05:09.395893+0000","last_peered":"2024-09-18T01:05:09.395893+0000","last_clean":"2024-09-18T01:05:09.395893+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:05:09.395893+0000","last_undegraded":"2024-09-18T01:05:09.395893+0000","last_fullsized":"2024-09-18T01:05:09.395893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"125'240","reported_seq":6067,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397675+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:05:09.397675+0000","last_peered":"2024-09-18T01:05:09.397675+0000","last_clean":"2024-09-18T01:05:09.397675+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:05:09.397675+0000","last_undegraded":"2024-09-18T01:05:09.397675+0000","last_fullsized":"2024-09-18T01:05:09.397675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"125'659","reported_seq":8756,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395703+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:05:09.395703+0000","last_peered":"2024-09-18T01:05:09.395703+0000","last_clean":"2024-09-18T01:05:09.395703+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:05:09.395703+0000","last_undegraded":"2024-09-18T01:05:09.395703+0000","last_fullsized":"2024-09-18T01:05:09.395703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"124'740","reported_seq":8844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"124'903","reported_seq":8734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396311+0000","last_change":"2024-09-18T01:04:57.304386+0000","last_active":"2024-09-18T01:05:09.396311+0000","last_peered":"2024-09-18T01:05:09.396311+0000","last_clean":"2024-09-18T01:05:09.396311+0000","last_became_active":"2024-09-18T01:04:57.302866+0000","last_became_peered":"2024-09-18T01:04:57.302866+0000","last_unstale":"2024-09-18T01:05:09.396311+0000","last_undegraded":"2024-09-18T01:05:09.396311+0000","last_fullsized":"2024-09-18T01:05:09.396311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:17.405000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"125'364","reported_seq":3882,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805054+0000","last_change":"2024-09-18T01:04:56.305126+0000","last_active":"2024-09-18T01:05:09.805054+0000","last_peered":"2024-09-18T01:05:09.805054+0000","last_clean":"2024-09-18T01:05:09.805054+0000","last_became_active":"2024-09-18T01:04:56.303131+0000","last_became_peered":"2024-09-18T01:04:56.303131+0000","last_unstale":"2024-09-18T01:05:09.805054+0000","last_undegraded":"2024-09-18T01:05:09.805054+0000","last_fullsized":"2024-09-18T01:05:09.805054+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:10.930635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"124'589","reported_seq":9916,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389745+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:05:08.389745+0000","last_peered":"2024-09-18T01:05:08.389745+0000","last_clean":"2024-09-18T01:05:08.389745+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:05:08.389745+0000","last_undegraded":"2024-09-18T01:05:08.389745+0000","last_fullsized":"2024-09-18T01:05:08.389745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"122'578","reported_seq":5352,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389651+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:05:08.389651+0000","last_peered":"2024-09-18T01:05:08.389651+0000","last_clean":"2024-09-18T01:05:08.389651+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:05:08.389651+0000","last_undegraded":"2024-09-18T01:05:08.389651+0000","last_fullsized":"2024-09-18T01:05:08.389651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389731+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:05:08.389731+0000","last_peered":"2024-09-18T01:05:08.389731+0000","last_clean":"2024-09-18T01:05:08.389731+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:05:08.389731+0000","last_undegraded":"2024-09-18T01:05:08.389731+0000","last_fullsized":"2024-09-18T01:05:08.389731+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7163,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395949+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:05:09.395949+0000","last_peered":"2024-09-18T01:05:09.395949+0000","last_clean":"2024-09-18T01:05:09.395949+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:05:09.395949+0000","last_undegraded":"2024-09-18T01:05:09.395949+0000","last_fullsized":"2024-09-18T01:05:09.395949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396147+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:05:09.396147+0000","last_peered":"2024-09-18T01:05:09.396147+0000","last_clean":"2024-09-18T01:05:09.396147+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:05:09.396147+0000","last_undegraded":"2024-09-18T01:05:09.396147+0000","last_fullsized":"2024-09-18T01:05:09.396147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10942,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929355+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:05:09.929355+0000","last_peered":"2024-09-18T01:05:09.929355+0000","last_clean":"2024-09-18T01:05:09.929355+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:05:09.929355+0000","last_undegraded":"2024-09-18T01:05:09.929355+0000","last_fullsized":"2024-09-18T01:05:09.929355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389817+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:05:08.389817+0000","last_peered":"2024-09-18T01:05:08.389817+0000","last_clean":"2024-09-18T01:05:08.389817+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:05:08.389817+0000","last_undegraded":"2024-09-18T01:05:08.389817+0000","last_fullsized":"2024-09-18T01:05:08.389817+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6372,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396101+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:05:09.396101+0000","last_peered":"2024-09-18T01:05:09.396101+0000","last_clean":"2024-09-18T01:05:09.396101+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:05:09.396101+0000","last_undegraded":"2024-09-18T01:05:09.396101+0000","last_fullsized":"2024-09-18T01:05:09.396101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396210+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:05:09.396210+0000","last_peered":"2024-09-18T01:05:09.396210+0000","last_clean":"2024-09-18T01:05:09.396210+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:05:09.396210+0000","last_undegraded":"2024-09-18T01:05:09.396210+0000","last_fullsized":"2024-09-18T01:05:09.396210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805037+0000","last_change":"2024-09-18T01:04:56.307373+0000","last_active":"2024-09-18T01:05:09.805037+0000","last_peered":"2024-09-18T01:05:09.805037+0000","last_clean":"2024-09-18T01:05:09.805037+0000","last_became_active":"2024-09-18T01:04:56.305312+0000","last_became_peered":"2024-09-18T01:04:56.305312+0000","last_unstale":"2024-09-18T01:05:09.805037+0000","last_undegraded":"2024-09-18T01:05:09.805037+0000","last_fullsized":"2024-09-18T01:05:09.805037+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:50.866986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"125'1081","reported_seq":7538,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397772+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:05:09.397772+0000","last_peered":"2024-09-18T01:05:09.397772+0000","last_clean":"2024-09-18T01:05:09.397772+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:05:09.397772+0000","last_undegraded":"2024-09-18T01:05:09.397772+0000","last_fullsized":"2024-09-18T01:05:09.397772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804885+0000","last_change":"2024-09-18T01:04:53.308070+0000","last_active":"2024-09-18T01:05:09.804885+0000","last_peered":"2024-09-18T01:05:09.804885+0000","last_clean":"2024-09-18T01:05:09.804885+0000","last_became_active":"2024-09-18T01:04:53.302064+0000","last_became_peered":"2024-09-18T01:04:53.302064+0000","last_unstale":"2024-09-18T01:05:09.804885+0000","last_undegraded":"2024-09-18T01:05:09.804885+0000","last_fullsized":"2024-09-18T01:05:09.804885+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:39.560060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6751,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395474+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:05:09.395474+0000","last_peered":"2024-09-18T01:05:09.395474+0000","last_clean":"2024-09-18T01:05:09.395474+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:05:09.395474+0000","last_undegraded":"2024-09-18T01:05:09.395474+0000","last_fullsized":"2024-09-18T01:05:09.395474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929251+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:05:09.929251+0000","last_peered":"2024-09-18T01:05:09.929251+0000","last_clean":"2024-09-18T01:05:09.929251+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:05:09.929251+0000","last_undegraded":"2024-09-18T01:05:09.929251+0000","last_fullsized":"2024-09-18T01:05:09.929251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4599,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395962+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:05:09.395962+0000","last_peered":"2024-09-18T01:05:09.395962+0000","last_clean":"2024-09-18T01:05:09.395962+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:05:09.395962+0000","last_undegraded":"2024-09-18T01:05:09.395962+0000","last_fullsized":"2024-09-18T01:05:09.395962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":278,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389506+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:05:08.389506+0000","last_peered":"2024-09-18T01:05:08.389506+0000","last_clean":"2024-09-18T01:05:08.389506+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:05:08.389506+0000","last_undegraded":"2024-09-18T01:05:08.389506+0000","last_fullsized":"2024-09-18T01:05:08.389506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7250,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395678+0000","last_change":"2024-09-18T01:04:57.305044+0000","last_active":"2024-09-18T01:05:09.395678+0000","last_peered":"2024-09-18T01:05:09.395678+0000","last_clean":"2024-09-18T01:05:09.395678+0000","last_became_active":"2024-09-18T01:04:57.303668+0000","last_became_peered":"2024-09-18T01:04:57.303668+0000","last_unstale":"2024-09-18T01:05:09.395678+0000","last_undegraded":"2024-09-18T01:05:09.395678+0000","last_fullsized":"2024-09-18T01:05:09.395678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:29.198158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804899+0000","last_change":"2024-09-18T01:04:53.301659+0000","last_active":"2024-09-18T01:05:09.804899+0000","last_peered":"2024-09-18T01:05:09.804899+0000","last_clean":"2024-09-18T01:05:09.804899+0000","last_became_active":"2024-09-18T01:04:53.298761+0000","last_became_peered":"2024-09-18T01:04:53.298761+0000","last_unstale":"2024-09-18T01:05:09.804899+0000","last_undegraded":"2024-09-18T01:05:09.804899+0000","last_fullsized":"2024-09-18T01:05:09.804899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:51.494813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":430,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389519+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:05:08.389519+0000","last_peered":"2024-09-18T01:05:08.389519+0000","last_clean":"2024-09-18T01:05:08.389519+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:05:08.389519+0000","last_undegraded":"2024-09-18T01:05:08.389519+0000","last_fullsized":"2024-09-18T01:05:08.389519+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8401,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'444","reported_seq":5116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395593+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:05:09.395593+0000","last_peered":"2024-09-18T01:05:09.395593+0000","last_clean":"2024-09-18T01:05:09.395593+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:05:09.395593+0000","last_undegraded":"2024-09-18T01:05:09.395593+0000","last_fullsized":"2024-09-18T01:05:09.395593+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"123'766","reported_seq":5462,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389682+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:05:08.389682+0000","last_peered":"2024-09-18T01:05:08.389682+0000","last_clean":"2024-09-18T01:05:08.389682+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:05:08.389682+0000","last_undegraded":"2024-09-18T01:05:08.389682+0000","last_fullsized":"2024-09-18T01:05:08.389682+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'511","reported_seq":6976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395961+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:05:09.395961+0000","last_peered":"2024-09-18T01:05:09.395961+0000","last_clean":"2024-09-18T01:05:09.395961+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:05:09.395961+0000","last_undegraded":"2024-09-18T01:05:09.395961+0000","last_fullsized":"2024-09-18T01:05:09.395961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"114'371","reported_seq":6081,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804974+0000","last_change":"2024-09-18T01:04:56.306988+0000","last_active":"2024-09-18T01:05:09.804974+0000","last_peered":"2024-09-18T01:05:09.804974+0000","last_clean":"2024-09-18T01:05:09.804974+0000","last_became_active":"2024-09-18T01:04:56.304292+0000","last_became_peered":"2024-09-18T01:04:56.304292+0000","last_unstale":"2024-09-18T01:05:09.804974+0000","last_undegraded":"2024-09-18T01:05:09.804974+0000","last_fullsized":"2024-09-18T01:05:09.804974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:54.467271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'530","reported_seq":6215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389683+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:05:08.389683+0000","last_peered":"2024-09-18T01:05:08.389683+0000","last_clean":"2024-09-18T01:05:08.389683+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:05:08.389683+0000","last_undegraded":"2024-09-18T01:05:08.389683+0000","last_fullsized":"2024-09-18T01:05:08.389683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'399","reported_seq":6889,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805010+0000","last_change":"2024-09-18T01:04:56.308511+0000","last_active":"2024-09-18T01:05:09.805010+0000","last_peered":"2024-09-18T01:05:09.805010+0000","last_clean":"2024-09-18T01:05:09.805010+0000","last_became_active":"2024-09-18T01:04:56.306823+0000","last_became_peered":"2024-09-18T01:04:56.306823+0000","last_unstale":"2024-09-18T01:05:09.805010+0000","last_undegraded":"2024-09-18T01:05:09.805010+0000","last_fullsized":"2024-09-18T01:05:09.805010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:11:50.012365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"119'614","reported_seq":13226,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805091+0000","last_change":"2024-09-18T01:04:56.305438+0000","last_active":"2024-09-18T01:05:09.805091+0000","last_peered":"2024-09-18T01:05:09.805091+0000","last_clean":"2024-09-18T01:05:09.805091+0000","last_became_active":"2024-09-18T01:04:56.303159+0000","last_became_peered":"2024-09-18T01:04:56.303159+0000","last_unstale":"2024-09-18T01:05:09.805091+0000","last_undegraded":"2024-09-18T01:05:09.805091+0000","last_fullsized":"2024-09-18T01:05:09.805091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:12.250901+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"122'439","reported_seq":4641,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804829+0000","last_change":"2024-09-18T01:04:56.307127+0000","last_active":"2024-09-18T01:05:09.804829+0000","last_peered":"2024-09-18T01:05:09.804829+0000","last_clean":"2024-09-18T01:05:09.804829+0000","last_became_active":"2024-09-18T01:04:56.303327+0000","last_became_peered":"2024-09-18T01:04:56.303327+0000","last_unstale":"2024-09-18T01:05:09.804829+0000","last_undegraded":"2024-09-18T01:05:09.804829+0000","last_fullsized":"2024-09-18T01:05:09.804829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:45.602917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"121'408","reported_seq":4531,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389521+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:05:08.389521+0000","last_peered":"2024-09-18T01:05:08.389521+0000","last_clean":"2024-09-18T01:05:08.389521+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:05:08.389521+0000","last_undegraded":"2024-09-18T01:05:08.389521+0000","last_fullsized":"2024-09-18T01:05:08.389521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'976","reported_seq":10504,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395722+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:05:09.395722+0000","last_peered":"2024-09-18T01:05:09.395722+0000","last_clean":"2024-09-18T01:05:09.395722+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:05:09.395722+0000","last_undegraded":"2024-09-18T01:05:09.395722+0000","last_fullsized":"2024-09-18T01:05:09.395722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"124'313","reported_seq":4114,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-18T01:05:08.389563+0000","last_change":"2024-09-18T01:04:56.304156+0000","last_active":"2024-09-18T01:05:08.389563+0000","last_peered":"2024-09-18T01:05:08.389563+0000","last_clean":"2024-09-18T01:05:08.389563+0000","last_became_active":"2024-09-18T01:04:56.302746+0000","last_became_peered":"2024-09-18T01:04:56.302746+0000","last_unstale":"2024-09-18T01:05:08.389563+0000","last_undegraded":"2024-09-18T01:05:08.389563+0000","last_fullsized":"2024-09-18T01:05:08.389563+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:41:30.202654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265609,"num_read_kb":1200484,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640292,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":1761280,"data_stored":725496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1814528,"data_stored":1315845,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640292,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77324,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294516,"statfs":{"total":96636764160,"available":96557584384,"internally_reserved":0,"allocated":1757184,"data_stored":725477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640294,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64868,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306972,"statfs":{"total":96636764160,"available":96570339328,"internally_reserved":0,"allocated":1777664,"data_stored":730485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640293,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46864,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324976,"statfs":{"total":96636764160,"available":96588775424,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:05:15.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T01:05:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:15.650 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":148,"fsid":"62b46b83-8ef2-466f-b28b-2d97f265422e","created":"2024-09-18T01:02:13.493841+0000","modified":"2024-09-18T01:05:09.391503+0000","last_up_change":"2024-09-18T01:02:24.290620+0000","last_in_change":"2024-09-18T01:04:52.444874+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-18T01:02:24.388064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-18T01:02:30.676199+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T01:02:40.390732+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4563fb16-8745-4e75-b654-1e96403d91b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6817","nonce":1626166637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6819","nonce":1626166637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6823","nonce":1626166637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1626166637},{"type":"v1","addr":"172.21.15.31:6821","nonce":1626166637}]},"public_addr":"172.21.15.31:6817/1626166637","cluster_addr":"172.21.15.31:6819/1626166637","heartbeat_back_addr":"172.21.15.31:6823/1626166637","heartbeat_front_addr":"172.21.15.31:6821/1626166637","state":["exists","up"]},{"osd":1,"uuid":"634965cb-d444-4721-b34a-afbc49105cab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6801","nonce":2210970259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6803","nonce":2210970259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6807","nonce":2210970259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2210970259},{"type":"v1","addr":"172.21.15.31:6805","nonce":2210970259}]},"public_addr":"172.21.15.31:6801/2210970259","cluster_addr":"172.21.15.31:6803/2210970259","heartbeat_back_addr":"172.21.15.31:6807/2210970259","heartbeat_front_addr":"172.21.15.31:6805/2210970259","state":["exists","up"]},{"osd":2,"uuid":"83443eab-2fa4-4c7e-a459-edb1bb0ed8d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6809","nonce":617133675}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6811","nonce":617133675}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6815","nonce":617133675}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":617133675},{"type":"v1","addr":"172.21.15.31:6813","nonce":617133675}]},"public_addr":"172.21.15.31:6809/617133675","cluster_addr":"172.21.15.31:6811/617133675","heartbeat_back_addr":"172.21.15.31:6815/617133675","heartbeat_front_addr":"172.21.15.31:6813/617133675","state":["exists","up"]},{"osd":3,"uuid":"5797982a-71c1-4d6a-9604-00fbc6bd16d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6809","nonce":151031464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6811","nonce":151031464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6815","nonce":151031464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":151031464},{"type":"v1","addr":"172.21.15.98:6813","nonce":151031464}]},"public_addr":"172.21.15.98:6809/151031464","cluster_addr":"172.21.15.98:6811/151031464","heartbeat_back_addr":"172.21.15.98:6815/151031464","heartbeat_front_addr":"172.21.15.98:6813/151031464","state":["exists","up"]},{"osd":4,"uuid":"f79e3329-5481-4f33-8d28-6386af186492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6817","nonce":1499036965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6819","nonce":1499036965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6823","nonce":1499036965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1499036965},{"type":"v1","addr":"172.21.15.98:6821","nonce":1499036965}]},"public_addr":"172.21.15.98:6817/1499036965","cluster_addr":"172.21.15.98:6819/1499036965","heartbeat_back_addr":"172.21.15.98:6823/1499036965","heartbeat_front_addr":"172.21.15.98:6821/1499036965","state":["exists","up"]},{"osd":5,"uuid":"8c61d2ec-b111-4783-a016-336ed33a0694","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6801","nonce":2310498524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6803","nonce":2310498524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6807","nonce":2310498524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2310498524},{"type":"v1","addr":"172.21.15.98:6805","nonce":2310498524}]},"public_addr":"172.21.15.98:6801/2310498524","cluster_addr":"172.21.15.98:6803/2310498524","heartbeat_back_addr":"172.21.15.98:6807/2310498524","heartbeat_front_addr":"172.21.15.98:6805/2310498524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.877700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.360930+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.403205+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.952123+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:22.099545+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T01:02:21.694132+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.9","osds":[1,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T01:05:16.663 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T01:05:16.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:05:16.798 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:05:16.798 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:05:16.798 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:05:16.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T01:05:17.100 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T01:05:17.113 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T01:05:17.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:05:17.238 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:05:17.238 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:05:17.239 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:05:17.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T01:05:17.541 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T01:05:17.554 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T01:05:17.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:05:17.680 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:05:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:05:17.680 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:05:17.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T01:05:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T01:05:17.993 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T01:05:17.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:05:18.120 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:05:18.120 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:05:18.120 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:05:18.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T01:05:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T01:05:18.435 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T01:05:18.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:05:18.562 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:05:18.562 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:05:18.563 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:05:18.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T01:05:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T01:05:18.876 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T01:05:18.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T01:05:19.002 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-18T01:05:19.002 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T01:05:19.002 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-18T01:05:19.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T01:05:19.304 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T01:05:19.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:05:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:19.612 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:05:19.623 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-18T01:05:18.372683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265726,"num_read_kb":1200585,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20721,"ondisk_log_size":20721,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":323992,"kb_used_data":10496,"kb_used_omap":9,"kb_used_meta":313334,"kb_avail":565907048,"statfs":{"total":579820584960,"available":579488817152,"internally_reserved":0,"allocated":10747904,"data_stored":5563494,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":320854688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.984469"},"pg_stats":[{"pgid":"3.1f","version":"125'658","reported_seq":10557,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396239+0000","last_change":"2024-09-18T01:04:46.431514+0000","last_active":"2024-09-18T01:05:09.396239+0000","last_peered":"2024-09-18T01:05:09.396239+0000","last_clean":"2024-09-18T01:05:09.396239+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:05:09.396239+0000","last_undegraded":"2024-09-18T01:05:09.396239+0000","last_fullsized":"2024-09-18T01:05:09.396239+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:28.663784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"114'1473","reported_seq":7752,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929303+0000","last_change":"2024-09-18T01:04:46.428713+0000","last_active":"2024-09-18T01:05:09.929303+0000","last_peered":"2024-09-18T01:05:09.929303+0000","last_clean":"2024-09-18T01:05:09.929303+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:05:09.929303+0000","last_undegraded":"2024-09-18T01:05:09.929303+0000","last_fullsized":"2024-09-18T01:05:09.929303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1473,"log_dups_size":0,"ondisk_log_size":1473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:40.544122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"124'455","reported_seq":7452,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805024+0000","last_change":"2024-09-18T01:04:56.309151+0000","last_active":"2024-09-18T01:05:09.805024+0000","last_peered":"2024-09-18T01:05:09.805024+0000","last_clean":"2024-09-18T01:05:09.805024+0000","last_became_active":"2024-09-18T01:04:56.307394+0000","last_became_peered":"2024-09-18T01:04:56.307394+0000","last_unstale":"2024-09-18T01:05:09.805024+0000","last_undegraded":"2024-09-18T01:05:09.805024+0000","last_fullsized":"2024-09-18T01:05:09.805024+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:35.044114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020314901999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"125'869","reported_seq":12809,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804894+0000","last_change":"2024-09-18T01:04:57.305367+0000","last_active":"2024-09-18T01:05:09.804894+0000","last_peered":"2024-09-18T01:05:09.804894+0000","last_clean":"2024-09-18T01:05:09.804894+0000","last_became_active":"2024-09-18T01:04:57.303594+0000","last_became_peered":"2024-09-18T01:04:57.303594+0000","last_unstale":"2024-09-18T01:05:09.804894+0000","last_undegraded":"2024-09-18T01:05:09.804894+0000","last_fullsized":"2024-09-18T01:05:09.804894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:51.883218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15908,"num_read_kb":43612,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"124'637","reported_seq":4531,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395893+0000","last_change":"2024-09-18T01:04:46.431924+0000","last_active":"2024-09-18T01:05:09.395893+0000","last_peered":"2024-09-18T01:05:09.395893+0000","last_clean":"2024-09-18T01:05:09.395893+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:05:09.395893+0000","last_undegraded":"2024-09-18T01:05:09.395893+0000","last_fullsized":"2024-09-18T01:05:09.395893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:42.672691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"125'240","reported_seq":6068,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397755+0000","last_change":"2024-09-18T01:04:46.426327+0000","last_active":"2024-09-18T01:05:09.397755+0000","last_peered":"2024-09-18T01:05:09.397755+0000","last_clean":"2024-09-18T01:05:09.397755+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:05:09.397755+0000","last_undegraded":"2024-09-18T01:05:09.397755+0000","last_fullsized":"2024-09-18T01:05:09.397755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:44.070641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"125'659","reported_seq":8756,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395703+0000","last_change":"2024-09-18T01:04:46.443599+0000","last_active":"2024-09-18T01:05:09.395703+0000","last_peered":"2024-09-18T01:05:09.395703+0000","last_clean":"2024-09-18T01:05:09.395703+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:05:09.395703+0000","last_undegraded":"2024-09-18T01:05:09.395703+0000","last_fullsized":"2024-09-18T01:05:09.395703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:39.030997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"124'740","reported_seq":8844,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.428082+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:16:52.180264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"124'903","reported_seq":8734,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396311+0000","last_change":"2024-09-18T01:04:57.304386+0000","last_active":"2024-09-18T01:05:09.396311+0000","last_peered":"2024-09-18T01:05:09.396311+0000","last_clean":"2024-09-18T01:05:09.396311+0000","last_became_active":"2024-09-18T01:04:57.302866+0000","last_became_peered":"2024-09-18T01:04:57.302866+0000","last_unstale":"2024-09-18T01:05:09.396311+0000","last_undegraded":"2024-09-18T01:05:09.396311+0000","last_fullsized":"2024-09-18T01:05:09.396311+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:17.405000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"125'364","reported_seq":3882,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805054+0000","last_change":"2024-09-18T01:04:56.305126+0000","last_active":"2024-09-18T01:05:09.805054+0000","last_peered":"2024-09-18T01:05:09.805054+0000","last_clean":"2024-09-18T01:05:09.805054+0000","last_became_active":"2024-09-18T01:04:56.303131+0000","last_became_peered":"2024-09-18T01:04:56.303131+0000","last_unstale":"2024-09-18T01:05:09.805054+0000","last_undegraded":"2024-09-18T01:05:09.805054+0000","last_fullsized":"2024-09-18T01:05:09.805054+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:10.930635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"124'589","reported_seq":9918,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775470+0000","last_change":"2024-09-18T01:04:46.501235+0000","last_active":"2024-09-18T01:05:10.775470+0000","last_peered":"2024-09-18T01:05:10.775470+0000","last_clean":"2024-09-18T01:05:10.775470+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:05:10.775470+0000","last_undegraded":"2024-09-18T01:05:10.775470+0000","last_fullsized":"2024-09-18T01:05:10.775470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:11.008700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"122'578","reported_seq":5354,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775441+0000","last_change":"2024-09-18T01:04:46.500737+0000","last_active":"2024-09-18T01:05:10.775441+0000","last_peered":"2024-09-18T01:05:10.775441+0000","last_clean":"2024-09-18T01:05:10.775441+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:05:10.775441+0000","last_undegraded":"2024-09-18T01:05:10.775441+0000","last_fullsized":"2024-09-18T01:05:10.775441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:09.987477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775739+0000","last_change":"2024-09-18T01:02:34.427826+0000","last_active":"2024-09-18T01:05:10.775739+0000","last_peered":"2024-09-18T01:05:10.775739+0000","last_clean":"2024-09-18T01:05:10.775739+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:05:10.775739+0000","last_undegraded":"2024-09-18T01:05:10.775739+0000","last_fullsized":"2024-09-18T01:05:10.775739+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:01.937179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'771","reported_seq":7163,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395949+0000","last_change":"2024-09-18T01:04:46.431550+0000","last_active":"2024-09-18T01:05:09.395949+0000","last_peered":"2024-09-18T01:05:09.395949+0000","last_clean":"2024-09-18T01:05:09.395949+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:05:09.395949+0000","last_undegraded":"2024-09-18T01:05:09.395949+0000","last_fullsized":"2024-09-18T01:05:09.395949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:59.241717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396147+0000","last_change":"2024-09-18T01:02:34.384115+0000","last_active":"2024-09-18T01:05:09.396147+0000","last_peered":"2024-09-18T01:05:09.396147+0000","last_clean":"2024-09-18T01:05:09.396147+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:05:09.396147+0000","last_undegraded":"2024-09-18T01:05:09.396147+0000","last_fullsized":"2024-09-18T01:05:09.396147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:53.774171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"124'2112","reported_seq":10942,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929355+0000","last_change":"2024-09-18T01:04:46.428447+0000","last_active":"2024-09-18T01:05:09.929355+0000","last_peered":"2024-09-18T01:05:09.929355+0000","last_clean":"2024-09-18T01:05:09.929355+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:05:09.929355+0000","last_undegraded":"2024-09-18T01:05:09.929355+0000","last_fullsized":"2024-09-18T01:05:09.929355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:13:06.340706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775643+0000","last_change":"2024-09-18T01:02:34.428134+0000","last_active":"2024-09-18T01:05:10.775643+0000","last_peered":"2024-09-18T01:05:10.775643+0000","last_clean":"2024-09-18T01:05:10.775643+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:05:10.775643+0000","last_undegraded":"2024-09-18T01:05:10.775643+0000","last_fullsized":"2024-09-18T01:05:10.775643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:16:44.027775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"124'488","reported_seq":6372,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396101+0000","last_change":"2024-09-18T01:04:46.443008+0000","last_active":"2024-09-18T01:05:09.396101+0000","last_peered":"2024-09-18T01:05:09.396101+0000","last_clean":"2024-09-18T01:05:09.396101+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:05:09.396101+0000","last_undegraded":"2024-09-18T01:05:09.396101+0000","last_fullsized":"2024-09-18T01:05:09.396101+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:55.596812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.396210+0000","last_change":"2024-09-18T01:02:34.383996+0000","last_active":"2024-09-18T01:05:09.396210+0000","last_peered":"2024-09-18T01:05:09.396210+0000","last_clean":"2024-09-18T01:05:09.396210+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:05:09.396210+0000","last_undegraded":"2024-09-18T01:05:09.396210+0000","last_fullsized":"2024-09-18T01:05:09.396210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:58:32.073056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"122'361","reported_seq":8179,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805037+0000","last_change":"2024-09-18T01:04:56.307373+0000","last_active":"2024-09-18T01:05:09.805037+0000","last_peered":"2024-09-18T01:05:09.805037+0000","last_clean":"2024-09-18T01:05:09.805037+0000","last_became_active":"2024-09-18T01:04:56.305312+0000","last_became_peered":"2024-09-18T01:04:56.305312+0000","last_unstale":"2024-09-18T01:05:09.805037+0000","last_undegraded":"2024-09-18T01:05:09.805037+0000","last_fullsized":"2024-09-18T01:05:09.805037+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:50.866986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"125'1081","reported_seq":7538,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.397772+0000","last_change":"2024-09-18T01:04:46.426834+0000","last_active":"2024-09-18T01:05:09.397772+0000","last_peered":"2024-09-18T01:05:09.397772+0000","last_clean":"2024-09-18T01:05:09.397772+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:05:09.397772+0000","last_undegraded":"2024-09-18T01:05:09.397772+0000","last_fullsized":"2024-09-18T01:05:09.397772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:11:19.946000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804885+0000","last_change":"2024-09-18T01:04:53.308070+0000","last_active":"2024-09-18T01:05:09.804885+0000","last_peered":"2024-09-18T01:05:09.804885+0000","last_clean":"2024-09-18T01:05:09.804885+0000","last_became_active":"2024-09-18T01:04:53.302064+0000","last_became_peered":"2024-09-18T01:04:53.302064+0000","last_unstale":"2024-09-18T01:05:09.804885+0000","last_undegraded":"2024-09-18T01:05:09.804885+0000","last_fullsized":"2024-09-18T01:05:09.804885+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:40:39.560060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"124'306","reported_seq":6751,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395474+0000","last_change":"2024-09-18T01:04:46.442435+0000","last_active":"2024-09-18T01:05:09.395474+0000","last_peered":"2024-09-18T01:05:09.395474+0000","last_clean":"2024-09-18T01:05:09.395474+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:05:09.395474+0000","last_undegraded":"2024-09-18T01:05:09.395474+0000","last_fullsized":"2024-09-18T01:05:09.395474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:40.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":279,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929251+0000","last_change":"2024-09-18T01:02:34.378746+0000","last_active":"2024-09-18T01:05:09.929251+0000","last_peered":"2024-09-18T01:05:09.929251+0000","last_clean":"2024-09-18T01:05:09.929251+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:05:09.929251+0000","last_undegraded":"2024-09-18T01:05:09.929251+0000","last_fullsized":"2024-09-18T01:05:09.929251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:41.858999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"124'375","reported_seq":4599,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395962+0000","last_change":"2024-09-18T01:04:46.431133+0000","last_active":"2024-09-18T01:05:09.395962+0000","last_peered":"2024-09-18T01:05:09.395962+0000","last_clean":"2024-09-18T01:05:09.395962+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:05:09.395962+0000","last_undegraded":"2024-09-18T01:05:09.395962+0000","last_fullsized":"2024-09-18T01:05:09.395962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:42.893234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":280,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775448+0000","last_change":"2024-09-18T01:02:34.427830+0000","last_active":"2024-09-18T01:05:10.775448+0000","last_peered":"2024-09-18T01:05:10.775448+0000","last_clean":"2024-09-18T01:05:10.775448+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:05:10.775448+0000","last_undegraded":"2024-09-18T01:05:10.775448+0000","last_fullsized":"2024-09-18T01:05:10.775448+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:53:33.938344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"124'487","reported_seq":7250,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395678+0000","last_change":"2024-09-18T01:04:57.305044+0000","last_active":"2024-09-18T01:05:09.395678+0000","last_peered":"2024-09-18T01:05:09.395678+0000","last_clean":"2024-09-18T01:05:09.395678+0000","last_became_active":"2024-09-18T01:04:57.303668+0000","last_became_peered":"2024-09-18T01:04:57.303668+0000","last_unstale":"2024-09-18T01:05:09.395678+0000","last_undegraded":"2024-09-18T01:05:09.395678+0000","last_fullsized":"2024-09-18T01:05:09.395678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:29.198158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":44,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804899+0000","last_change":"2024-09-18T01:04:53.301659+0000","last_active":"2024-09-18T01:05:09.804899+0000","last_peered":"2024-09-18T01:05:09.804899+0000","last_clean":"2024-09-18T01:05:09.804899+0000","last_became_active":"2024-09-18T01:04:53.298761+0000","last_became_peered":"2024-09-18T01:04:53.298761+0000","last_unstale":"2024-09-18T01:05:09.804899+0000","last_undegraded":"2024-09-18T01:05:09.804899+0000","last_fullsized":"2024-09-18T01:05:09.804899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:31.353487+0000","last_clean_scrub_stamp":"2024-09-18T01:02:31.353487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:51.494813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":432,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775437+0000","last_change":"2024-09-18T01:02:26.302174+0000","last_active":"2024-09-18T01:05:10.775437+0000","last_peered":"2024-09-18T01:05:10.775437+0000","last_clean":"2024-09-18T01:05:10.775437+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:05:10.775437+0000","last_undegraded":"2024-09-18T01:05:10.775437+0000","last_fullsized":"2024-09-18T01:05:10.775437+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:25.291962+0000","last_clean_scrub_stamp":"2024-09-18T01:02:25.291962+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:52:00.773534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"121'684","reported_seq":8401,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.929227+0000","last_change":"2024-09-18T01:04:46.427998+0000","last_active":"2024-09-18T01:05:09.929227+0000","last_peered":"2024-09-18T01:05:09.929227+0000","last_clean":"2024-09-18T01:05:09.929227+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:05:09.929227+0000","last_undegraded":"2024-09-18T01:05:09.929227+0000","last_fullsized":"2024-09-18T01:05:09.929227+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:17.698369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'444","reported_seq":5116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395593+0000","last_change":"2024-09-18T01:04:46.443657+0000","last_active":"2024-09-18T01:05:09.395593+0000","last_peered":"2024-09-18T01:05:09.395593+0000","last_clean":"2024-09-18T01:05:09.395593+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:05:09.395593+0000","last_undegraded":"2024-09-18T01:05:09.395593+0000","last_fullsized":"2024-09-18T01:05:09.395593+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:18.846930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"123'766","reported_seq":5464,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775535+0000","last_change":"2024-09-18T01:04:46.502091+0000","last_active":"2024-09-18T01:05:10.775535+0000","last_peered":"2024-09-18T01:05:10.775535+0000","last_clean":"2024-09-18T01:05:10.775535+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:05:10.775535+0000","last_undegraded":"2024-09-18T01:05:10.775535+0000","last_fullsized":"2024-09-18T01:05:10.775535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:58.258402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'511","reported_seq":6976,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395961+0000","last_change":"2024-09-18T01:04:46.442505+0000","last_active":"2024-09-18T01:05:09.395961+0000","last_peered":"2024-09-18T01:05:09.395961+0000","last_clean":"2024-09-18T01:05:09.395961+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:05:09.395961+0000","last_undegraded":"2024-09-18T01:05:09.395961+0000","last_fullsized":"2024-09-18T01:05:09.395961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:21.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"114'371","reported_seq":6081,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804974+0000","last_change":"2024-09-18T01:04:56.306988+0000","last_active":"2024-09-18T01:05:09.804974+0000","last_peered":"2024-09-18T01:05:09.804974+0000","last_clean":"2024-09-18T01:05:09.804974+0000","last_became_active":"2024-09-18T01:04:56.304292+0000","last_became_peered":"2024-09-18T01:04:56.304292+0000","last_unstale":"2024-09-18T01:05:09.804974+0000","last_undegraded":"2024-09-18T01:05:09.804974+0000","last_fullsized":"2024-09-18T01:05:09.804974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:54.467271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"120'530","reported_seq":6217,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775635+0000","last_change":"2024-09-18T01:04:46.501226+0000","last_active":"2024-09-18T01:05:10.775635+0000","last_peered":"2024-09-18T01:05:10.775635+0000","last_clean":"2024-09-18T01:05:10.775635+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:05:10.775635+0000","last_undegraded":"2024-09-18T01:05:10.775635+0000","last_fullsized":"2024-09-18T01:05:10.775635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:36.643398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'399","reported_seq":6889,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805010+0000","last_change":"2024-09-18T01:04:56.308511+0000","last_active":"2024-09-18T01:05:09.805010+0000","last_peered":"2024-09-18T01:05:09.805010+0000","last_clean":"2024-09-18T01:05:09.805010+0000","last_became_active":"2024-09-18T01:04:56.306823+0000","last_became_peered":"2024-09-18T01:04:56.306823+0000","last_unstale":"2024-09-18T01:05:09.805010+0000","last_undegraded":"2024-09-18T01:05:09.805010+0000","last_fullsized":"2024-09-18T01:05:09.805010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:11:50.012365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"119'614","reported_seq":13226,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.805091+0000","last_change":"2024-09-18T01:04:56.305438+0000","last_active":"2024-09-18T01:05:09.805091+0000","last_peered":"2024-09-18T01:05:09.805091+0000","last_clean":"2024-09-18T01:05:09.805091+0000","last_became_active":"2024-09-18T01:04:56.303159+0000","last_became_peered":"2024-09-18T01:04:56.303159+0000","last_unstale":"2024-09-18T01:05:09.805091+0000","last_undegraded":"2024-09-18T01:05:09.805091+0000","last_fullsized":"2024-09-18T01:05:09.805091+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:12.250901+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"122'439","reported_seq":4641,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.804829+0000","last_change":"2024-09-18T01:04:56.307127+0000","last_active":"2024-09-18T01:05:09.804829+0000","last_peered":"2024-09-18T01:05:09.804829+0000","last_clean":"2024-09-18T01:05:09.804829+0000","last_became_active":"2024-09-18T01:04:56.303327+0000","last_became_peered":"2024-09-18T01:04:56.303327+0000","last_unstale":"2024-09-18T01:05:09.804829+0000","last_undegraded":"2024-09-18T01:05:09.804829+0000","last_fullsized":"2024-09-18T01:05:09.804829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:09:45.602917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"121'408","reported_seq":4533,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775444+0000","last_change":"2024-09-18T01:04:46.500774+0000","last_active":"2024-09-18T01:05:10.775444+0000","last_peered":"2024-09-18T01:05:10.775444+0000","last_clean":"2024-09-18T01:05:10.775444+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:05:10.775444+0000","last_undegraded":"2024-09-18T01:05:10.775444+0000","last_fullsized":"2024-09-18T01:05:10.775444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:31:38.016987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"122'976","reported_seq":10504,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:09.395722+0000","last_change":"2024-09-18T01:04:46.430858+0000","last_active":"2024-09-18T01:05:09.395722+0000","last_peered":"2024-09-18T01:05:09.395722+0000","last_clean":"2024-09-18T01:05:09.395722+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:05:09.395722+0000","last_undegraded":"2024-09-18T01:05:09.395722+0000","last_fullsized":"2024-09-18T01:05:09.395722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":976,"log_dups_size":0,"ondisk_log_size":976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:27.639255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"124'313","reported_seq":4116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:10.775438+0000","last_change":"2024-09-18T01:04:56.304156+0000","last_active":"2024-09-18T01:05:10.775438+0000","last_peered":"2024-09-18T01:05:10.775438+0000","last_clean":"2024-09-18T01:05:10.775438+0000","last_became_active":"2024-09-18T01:04:56.302746+0000","last_became_peered":"2024-09-18T01:04:56.302746+0000","last_unstale":"2024-09-18T01:05:10.775438+0000","last_undegraded":"2024-09-18T01:05:10.775438+0000","last_fullsized":"2024-09-18T01:05:10.775438+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:02:40.440830+0000","last_clean_scrub_stamp":"2024-09-18T01:02:40.440830+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:41:30.202654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265609,"num_read_kb":1200484,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20601,"ondisk_log_size":20601,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640293,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640295,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46816,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325024,"statfs":{"total":96636764160,"available":96588824576,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51020,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320820,"statfs":{"total":96636764160,"available":96584519680,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640293,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77340,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294500,"statfs":{"total":96636764160,"available":96557568000,"internally_reserved":0,"allocated":1769472,"data_stored":730447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640295,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64868,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306972,"statfs":{"total":96636764160,"available":96570339328,"internally_reserved":0,"allocated":1777664,"data_stored":730485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640294,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46864,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324976,"statfs":{"total":96636764160,"available":96588775424,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:05:19.628 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.628 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.628 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.628 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.628 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.628 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.629 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.630 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-18T01:02:31.353487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=31, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T01:02:25.291962+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=25, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.631 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.632 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.633 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-18T01:02:40.440830+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=2, tm_sec=40, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=1, tm_min=5, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T01:05:19.633 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T01:05:39.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T01:05:39.938 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-18T01:05:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-18T01:05:39.950 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-18T01:05:38.377297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":175,"num_read":265726,"num_read_kb":1200585,"num_write":23088,"num_write_kb":288827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2359,"num_bytes_recovered":32233475,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20904,"ondisk_log_size":20904,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":328096,"kb_used_data":10496,"kb_used_omap":9,"kb_used_meta":317494,"kb_avail":565902944,"statfs":{"total":579820584960,"available":579484614656,"internally_reserved":0,"allocated":10747904,"data_stored":5563494,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":325114528},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002689"},"pg_stats":[{"pgid":"3.1f","version":"148'670","reported_seq":10578,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:20.295715+0000","last_change":"2024-09-18T01:05:20.295761+0000","last_active":"2024-09-18T01:05:20.295715+0000","last_peered":"2024-09-18T01:05:20.295715+0000","last_clean":"2024-09-18T01:05:20.295715+0000","last_became_active":"2024-09-18T01:04:36.371213+0000","last_became_peered":"2024-09-18T01:04:36.371213+0000","last_unstale":"2024-09-18T01:05:20.295715+0000","last_undegraded":"2024-09-18T01:05:20.295715+0000","last_fullsized":"2024-09-18T01:05:20.295715+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'670","last_scrub_stamp":"2024-09-18T01:05:20.295621+0000","last_deep_scrub":"148'670","last_deep_scrub_stamp":"2024-09-18T01:05:20.295621+0000","last_clean_scrub_stamp":"2024-09-18T01:05:20.295621+0000","objects_scrubbed":12,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:10:38.771736+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0012080299999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12635,"num_read_kb":58133,"num_write":543,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5264416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"148'1478","reported_seq":7766,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:20.560453+0000","last_change":"2024-09-18T01:05:20.560513+0000","last_active":"2024-09-18T01:05:20.560453+0000","last_peered":"2024-09-18T01:05:20.560453+0000","last_clean":"2024-09-18T01:05:20.560453+0000","last_became_active":"2024-09-18T01:02:41.457883+0000","last_became_peered":"2024-09-18T01:02:41.457883+0000","last_unstale":"2024-09-18T01:05:20.560453+0000","last_undegraded":"2024-09-18T01:05:20.560453+0000","last_fullsized":"2024-09-18T01:05:20.560453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'1478","last_scrub_stamp":"2024-09-18T01:05:20.560309+0000","last_deep_scrub":"148'1478","last_deep_scrub_stamp":"2024-09-18T01:05:20.560309+0000","last_clean_scrub_stamp":"2024-09-18T01:05:20.560309+0000","objects_scrubbed":5,"log_size":1478,"log_dups_size":0,"ondisk_log_size":1478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:56.311551+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00096842399999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":12260,"num_read_kb":29244,"num_write":2012,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"148'457","reported_seq":7463,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:27.411063+0000","last_change":"2024-09-18T01:05:27.411109+0000","last_active":"2024-09-18T01:05:27.411063+0000","last_peered":"2024-09-18T01:05:27.411063+0000","last_clean":"2024-09-18T01:05:27.411063+0000","last_became_active":"2024-09-18T01:04:56.307394+0000","last_became_peered":"2024-09-18T01:04:56.307394+0000","last_unstale":"2024-09-18T01:05:27.411063+0000","last_undegraded":"2024-09-18T01:05:27.411063+0000","last_fullsized":"2024-09-18T01:05:27.411063+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'457","last_scrub_stamp":"2024-09-18T01:05:27.410947+0000","last_deep_scrub":"148'457","last_deep_scrub_stamp":"2024-09-18T01:05:27.410947+0000","last_clean_scrub_stamp":"2024-09-18T01:05:27.410947+0000","objects_scrubbed":2,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:41:57.911080+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.020314901999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":7621,"num_read_kb":29178,"num_write":433,"num_write_kb":8552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"148'876","reported_seq":12825,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:28.376015+0000","last_change":"2024-09-18T01:05:28.376056+0000","last_active":"2024-09-18T01:05:28.376015+0000","last_peered":"2024-09-18T01:05:28.376015+0000","last_clean":"2024-09-18T01:05:28.376015+0000","last_became_active":"2024-09-18T01:04:57.303594+0000","last_became_peered":"2024-09-18T01:04:57.303594+0000","last_unstale":"2024-09-18T01:05:28.376015+0000","last_undegraded":"2024-09-18T01:05:28.376015+0000","last_fullsized":"2024-09-18T01:05:28.376015+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'876","last_scrub_stamp":"2024-09-18T01:05:28.375934+0000","last_deep_scrub":"148'876","last_deep_scrub_stamp":"2024-09-18T01:05:28.375934+0000","last_clean_scrub_stamp":"2024-09-18T01:05:28.375934+0000","objects_scrubbed":7,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:51.018720+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0012194479999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":15908,"num_read_kb":43612,"num_write":739,"num_write_kb":8649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":41,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"148'641","reported_seq":4544,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:21.293065+0000","last_change":"2024-09-18T01:05:21.293108+0000","last_active":"2024-09-18T01:05:21.293065+0000","last_peered":"2024-09-18T01:05:21.293065+0000","last_clean":"2024-09-18T01:05:21.293065+0000","last_became_active":"2024-09-18T01:04:39.297524+0000","last_became_peered":"2024-09-18T01:04:39.297524+0000","last_unstale":"2024-09-18T01:05:21.293065+0000","last_undegraded":"2024-09-18T01:05:21.293065+0000","last_fullsized":"2024-09-18T01:05:21.293065+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'641","last_scrub_stamp":"2024-09-18T01:05:21.292977+0000","last_deep_scrub":"148'641","last_deep_scrub_stamp":"2024-09-18T01:05:21.292977+0000","last_clean_scrub_stamp":"2024-09-18T01:05:21.292977+0000","objects_scrubbed":4,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:38:41.495191+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015148360000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5137,"num_read_kb":19596,"num_write":783,"num_write_kb":5934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":1681637,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"148'245","reported_seq":6082,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:19.602270+0000","last_change":"2024-09-18T01:05:19.602301+0000","last_active":"2024-09-18T01:05:19.602270+0000","last_peered":"2024-09-18T01:05:19.602270+0000","last_clean":"2024-09-18T01:05:19.602270+0000","last_became_active":"2024-09-18T01:02:41.455250+0000","last_became_peered":"2024-09-18T01:02:41.455250+0000","last_unstale":"2024-09-18T01:05:19.602270+0000","last_undegraded":"2024-09-18T01:05:19.602270+0000","last_fullsized":"2024-09-18T01:05:19.602270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'245","last_scrub_stamp":"2024-09-18T01:05:19.602167+0000","last_deep_scrub":"148'245","last_deep_scrub_stamp":"2024-09-18T01:05:19.602167+0000","last_clean_scrub_stamp":"2024-09-18T01:05:19.602167+0000","objects_scrubbed":5,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:54:36.581957+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00050116500000000003,"stat_sum":{"num_bytes":19,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":5453,"num_read_kb":19431,"num_write":232,"num_write_kb":9232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"148'667","reported_seq":8773,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:20.713813+0000","last_change":"2024-09-18T01:05:20.713913+0000","last_active":"2024-09-18T01:05:20.713813+0000","last_peered":"2024-09-18T01:05:20.713813+0000","last_clean":"2024-09-18T01:05:20.713813+0000","last_became_active":"2024-09-18T01:02:41.454819+0000","last_became_peered":"2024-09-18T01:02:41.454819+0000","last_unstale":"2024-09-18T01:05:20.713813+0000","last_undegraded":"2024-09-18T01:05:20.713813+0000","last_fullsized":"2024-09-18T01:05:20.713813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'667","last_scrub_stamp":"2024-09-18T01:05:20.713696+0000","last_deep_scrub":"148'667","last_deep_scrub_stamp":"2024-09-18T01:05:20.713696+0000","last_clean_scrub_stamp":"2024-09-18T01:05:20.713696+0000","objects_scrubbed":8,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:37:55.778892+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001512219,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11039,"num_read_kb":40135,"num_write":610,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"148'748","reported_seq":8861,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:21.598971+0000","last_change":"2024-09-18T01:05:21.599051+0000","last_active":"2024-09-18T01:05:21.598971+0000","last_peered":"2024-09-18T01:05:21.598971+0000","last_clean":"2024-09-18T01:05:21.598971+0000","last_became_active":"2024-09-18T01:04:39.301706+0000","last_became_peered":"2024-09-18T01:04:39.301706+0000","last_unstale":"2024-09-18T01:05:21.598971+0000","last_undegraded":"2024-09-18T01:05:21.598971+0000","last_fullsized":"2024-09-18T01:05:21.598971+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'748","last_scrub_stamp":"2024-09-18T01:05:21.598851+0000","last_deep_scrub":"148'748","last_deep_scrub_stamp":"2024-09-18T01:05:21.598851+0000","last_clean_scrub_stamp":"2024-09-18T01:05:21.598851+0000","objects_scrubbed":8,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:24.139138+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00066794100000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9266,"num_read_kb":83025,"num_write":705,"num_write_kb":22278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":11217461,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"148'912","reported_seq":8752,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:22.255117+0000","last_change":"2024-09-18T01:05:22.255162+0000","last_active":"2024-09-18T01:05:22.255117+0000","last_peered":"2024-09-18T01:05:22.255117+0000","last_clean":"2024-09-18T01:05:22.255117+0000","last_became_active":"2024-09-18T01:04:57.302866+0000","last_became_peered":"2024-09-18T01:04:57.302866+0000","last_unstale":"2024-09-18T01:05:22.255117+0000","last_undegraded":"2024-09-18T01:05:22.255117+0000","last_fullsized":"2024-09-18T01:05:22.255117+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'912","last_scrub_stamp":"2024-09-18T01:05:22.255010+0000","last_deep_scrub":"148'912","last_deep_scrub_stamp":"2024-09-18T01:05:22.255010+0000","last_clean_scrub_stamp":"2024-09-18T01:05:22.255010+0000","objects_scrubbed":9,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:48:22.885422+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001473869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10626,"num_read_kb":32381,"num_write":1050,"num_write_kb":6705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1811569,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"148'367","reported_seq":3894,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:21.375620+0000","last_change":"2024-09-18T01:05:21.375668+0000","last_active":"2024-09-18T01:05:21.375620+0000","last_peered":"2024-09-18T01:05:21.375620+0000","last_clean":"2024-09-18T01:05:21.375620+0000","last_became_active":"2024-09-18T01:04:56.303131+0000","last_became_peered":"2024-09-18T01:04:56.303131+0000","last_unstale":"2024-09-18T01:05:21.375620+0000","last_undegraded":"2024-09-18T01:05:21.375620+0000","last_fullsized":"2024-09-18T01:05:21.375620+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'367","last_scrub_stamp":"2024-09-18T01:05:21.375522+0000","last_deep_scrub":"148'367","last_deep_scrub_stamp":"2024-09-18T01:05:21.375522+0000","last_clean_scrub_stamp":"2024-09-18T01:05:21.375522+0000","objects_scrubbed":3,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:48:24.912166+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0016114,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3217,"num_read_kb":32434,"num_write":414,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7297295,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"148'598","reported_seq":9936,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:26.253845+0000","last_change":"2024-09-18T01:05:26.253886+0000","last_active":"2024-09-18T01:05:26.253845+0000","last_peered":"2024-09-18T01:05:26.253845+0000","last_clean":"2024-09-18T01:05:26.253845+0000","last_became_active":"2024-09-18T01:02:41.454865+0000","last_became_peered":"2024-09-18T01:02:41.454865+0000","last_unstale":"2024-09-18T01:05:26.253845+0000","last_undegraded":"2024-09-18T01:05:26.253845+0000","last_fullsized":"2024-09-18T01:05:26.253845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'598","last_scrub_stamp":"2024-09-18T01:05:26.253748+0000","last_deep_scrub":"148'598","last_deep_scrub_stamp":"2024-09-18T01:05:26.253748+0000","last_clean_scrub_stamp":"2024-09-18T01:05:26.253748+0000","objects_scrubbed":9,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:10.490376+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0013580840000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":11671,"num_read_kb":41436,"num_write":453,"num_write_kb":4769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"148'589","reported_seq":5374,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:27.254911+0000","last_change":"2024-09-18T01:05:27.254978+0000","last_active":"2024-09-18T01:05:27.254911+0000","last_peered":"2024-09-18T01:05:27.254911+0000","last_clean":"2024-09-18T01:05:27.254911+0000","last_became_active":"2024-09-18T01:02:41.454639+0000","last_became_peered":"2024-09-18T01:02:41.454639+0000","last_unstale":"2024-09-18T01:05:27.254911+0000","last_undegraded":"2024-09-18T01:05:27.254911+0000","last_fullsized":"2024-09-18T01:05:27.254911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'589","last_scrub_stamp":"2024-09-18T01:05:27.254807+0000","last_deep_scrub":"148'589","last_deep_scrub_stamp":"2024-09-18T01:05:27.254807+0000","last_clean_scrub_stamp":"2024-09-18T01:05:27.254807+0000","objects_scrubbed":11,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:24.735130+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00086979099999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6147,"num_read_kb":17374,"num_write":611,"num_write_kb":5090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":289,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:22.329481+0000","last_change":"2024-09-18T01:05:22.329508+0000","last_active":"2024-09-18T01:05:22.329481+0000","last_peered":"2024-09-18T01:05:22.329481+0000","last_clean":"2024-09-18T01:05:22.329481+0000","last_became_active":"2024-09-18T01:02:32.728158+0000","last_became_peered":"2024-09-18T01:02:32.728158+0000","last_unstale":"2024-09-18T01:05:22.329481+0000","last_undegraded":"2024-09-18T01:05:22.329481+0000","last_fullsized":"2024-09-18T01:05:22.329481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:05:22.329391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:05:22.329391+0000","last_clean_scrub_stamp":"2024-09-18T01:05:22.329391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:32:20.902379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036838399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"148'776","reported_seq":7177,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:19.254786+0000","last_change":"2024-09-18T01:05:19.254818+0000","last_active":"2024-09-18T01:05:19.254786+0000","last_peered":"2024-09-18T01:05:19.254786+0000","last_clean":"2024-09-18T01:05:19.254786+0000","last_became_active":"2024-09-18T01:02:41.455186+0000","last_became_peered":"2024-09-18T01:02:41.455186+0000","last_unstale":"2024-09-18T01:05:19.254786+0000","last_undegraded":"2024-09-18T01:05:19.254786+0000","last_fullsized":"2024-09-18T01:05:19.254786+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'776","last_scrub_stamp":"2024-09-18T01:05:19.254714+0000","last_deep_scrub":"148'776","last_deep_scrub_stamp":"2024-09-18T01:05:19.254714+0000","last_clean_scrub_stamp":"2024-09-18T01:05:19.254714+0000","objects_scrubbed":5,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:16.606566+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.001371887,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8034,"num_read_kb":75237,"num_write":974,"num_write_kb":12038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":289,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:17.676202+0000","last_change":"2024-09-18T01:05:17.676222+0000","last_active":"2024-09-18T01:05:17.676202+0000","last_peered":"2024-09-18T01:05:17.676202+0000","last_clean":"2024-09-18T01:05:17.676202+0000","last_became_active":"2024-09-18T01:02:32.372941+0000","last_became_peered":"2024-09-18T01:02:32.372941+0000","last_unstale":"2024-09-18T01:05:17.676202+0000","last_undegraded":"2024-09-18T01:05:17.676202+0000","last_fullsized":"2024-09-18T01:05:17.676202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:05:17.676134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:05:17.676134+0000","last_clean_scrub_stamp":"2024-09-18T01:05:17.676134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:40:40.018794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048864100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"148'2117","reported_seq":10956,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:22.561037+0000","last_change":"2024-09-18T01:05:22.561077+0000","last_active":"2024-09-18T01:05:22.561037+0000","last_peered":"2024-09-18T01:05:22.561037+0000","last_clean":"2024-09-18T01:05:22.561037+0000","last_became_active":"2024-09-18T01:04:41.396002+0000","last_became_peered":"2024-09-18T01:04:41.396002+0000","last_unstale":"2024-09-18T01:05:22.561037+0000","last_undegraded":"2024-09-18T01:05:22.561037+0000","last_fullsized":"2024-09-18T01:05:22.561037+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'2117","last_scrub_stamp":"2024-09-18T01:05:22.560926+0000","last_deep_scrub":"148'2117","last_deep_scrub_stamp":"2024-09-18T01:05:22.560926+0000","last_clean_scrub_stamp":"2024-09-18T01:05:22.560926+0000","objects_scrubbed":5,"log_size":2117,"log_dups_size":0,"ondisk_log_size":2117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:16:10.639157+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00070353099999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":19814,"num_read_kb":47264,"num_write":3363,"num_write_kb":9359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2321742,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":289,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:20.324059+0000","last_change":"2024-09-18T01:05:20.324081+0000","last_active":"2024-09-18T01:05:20.324059+0000","last_peered":"2024-09-18T01:05:20.324059+0000","last_clean":"2024-09-18T01:05:20.324059+0000","last_became_active":"2024-09-18T01:02:32.372436+0000","last_became_peered":"2024-09-18T01:02:32.372436+0000","last_unstale":"2024-09-18T01:05:20.324059+0000","last_undegraded":"2024-09-18T01:05:20.324059+0000","last_fullsized":"2024-09-18T01:05:20.324059+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:05:20.323975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:05:20.323975+0000","last_clean_scrub_stamp":"2024-09-18T01:05:20.323975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:58.004590+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00066996899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"148'489","reported_seq":6382,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:23.632586+0000","last_change":"2024-09-18T01:05:23.632623+0000","last_active":"2024-09-18T01:05:23.632586+0000","last_peered":"2024-09-18T01:05:23.632586+0000","last_clean":"2024-09-18T01:05:23.632586+0000","last_became_active":"2024-09-18T01:04:41.397372+0000","last_became_peered":"2024-09-18T01:04:41.397372+0000","last_unstale":"2024-09-18T01:05:23.632586+0000","last_undegraded":"2024-09-18T01:05:23.632586+0000","last_fullsized":"2024-09-18T01:05:23.632586+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'489","last_scrub_stamp":"2024-09-18T01:05:23.632481+0000","last_deep_scrub":"148'489","last_deep_scrub_stamp":"2024-09-18T01:05:23.632481+0000","last_clean_scrub_stamp":"2024-09-18T01:05:23.632481+0000","objects_scrubbed":1,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:24.517687+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00098432399999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5961,"num_read_kb":22104,"num_write":500,"num_write_kb":7404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":293972,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":289,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:18.663436+0000","last_change":"2024-09-18T01:05:18.663484+0000","last_active":"2024-09-18T01:05:18.663436+0000","last_peered":"2024-09-18T01:05:18.663436+0000","last_clean":"2024-09-18T01:05:18.663436+0000","last_became_active":"2024-09-18T01:02:32.374360+0000","last_became_peered":"2024-09-18T01:02:32.374360+0000","last_unstale":"2024-09-18T01:05:18.663436+0000","last_undegraded":"2024-09-18T01:05:18.663436+0000","last_fullsized":"2024-09-18T01:05:18.663436+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:05:18.663350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:05:18.663350+0000","last_clean_scrub_stamp":"2024-09-18T01:05:18.663350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:16.134305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"148'364","reported_seq":8191,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:25.468293+0000","last_change":"2024-09-18T01:05:25.468337+0000","last_active":"2024-09-18T01:05:25.468293+0000","last_peered":"2024-09-18T01:05:25.468293+0000","last_clean":"2024-09-18T01:05:25.468293+0000","last_became_active":"2024-09-18T01:04:56.305312+0000","last_became_peered":"2024-09-18T01:04:56.305312+0000","last_unstale":"2024-09-18T01:05:25.468293+0000","last_undegraded":"2024-09-18T01:05:25.468293+0000","last_fullsized":"2024-09-18T01:05:25.468293+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'364","last_scrub_stamp":"2024-09-18T01:05:25.468215+0000","last_deep_scrub":"148'364","last_deep_scrub_stamp":"2024-09-18T01:05:25.468215+0000","last_clean_scrub_stamp":"2024-09-18T01:05:25.468215+0000","objects_scrubbed":3,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:49.980781+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00077913699999999995,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7496,"num_read_kb":41290,"num_write":363,"num_write_kb":13049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"148'1087","reported_seq":7553,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:20.606338+0000","last_change":"2024-09-18T01:05:20.606374+0000","last_active":"2024-09-18T01:05:20.606338+0000","last_peered":"2024-09-18T01:05:20.606338+0000","last_clean":"2024-09-18T01:05:20.606338+0000","last_became_active":"2024-09-18T01:02:41.460411+0000","last_became_peered":"2024-09-18T01:02:41.460411+0000","last_unstale":"2024-09-18T01:05:20.606338+0000","last_undegraded":"2024-09-18T01:05:20.606338+0000","last_fullsized":"2024-09-18T01:05:20.606338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'1087","last_scrub_stamp":"2024-09-18T01:05:20.606256+0000","last_deep_scrub":"148'1087","last_deep_scrub_stamp":"2024-09-18T01:05:20.606256+0000","last_clean_scrub_stamp":"2024-09-18T01:05:20.606256+0000","objects_scrubbed":6,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:38:33.087782+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00097375499999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":9872,"num_read_kb":61655,"num_write":1350,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"148'3","reported_seq":54,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:20.355087+0000","last_change":"2024-09-18T01:05:20.355120+0000","last_active":"2024-09-18T01:05:20.355087+0000","last_peered":"2024-09-18T01:05:20.355087+0000","last_clean":"2024-09-18T01:05:20.355087+0000","last_became_active":"2024-09-18T01:04:53.302064+0000","last_became_peered":"2024-09-18T01:04:53.302064+0000","last_unstale":"2024-09-18T01:05:20.355087+0000","last_undegraded":"2024-09-18T01:05:20.355087+0000","last_fullsized":"2024-09-18T01:05:20.355087+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'3","last_scrub_stamp":"2024-09-18T01:05:20.354993+0000","last_deep_scrub":"148'3","last_deep_scrub_stamp":"2024-09-18T01:05:20.354993+0000","last_clean_scrub_stamp":"2024-09-18T01:05:20.354993+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:06.557463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"148'307","reported_seq":6761,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:19.705095+0000","last_change":"2024-09-18T01:05:19.705122+0000","last_active":"2024-09-18T01:05:19.705095+0000","last_peered":"2024-09-18T01:05:19.705095+0000","last_clean":"2024-09-18T01:05:19.705095+0000","last_became_active":"2024-09-18T01:02:41.454040+0000","last_became_peered":"2024-09-18T01:02:41.454040+0000","last_unstale":"2024-09-18T01:05:19.705095+0000","last_undegraded":"2024-09-18T01:05:19.705095+0000","last_fullsized":"2024-09-18T01:05:19.705095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'307","last_scrub_stamp":"2024-09-18T01:05:19.705027+0000","last_deep_scrub":"148'307","last_deep_scrub_stamp":"2024-09-18T01:05:19.705027+0000","last_clean_scrub_stamp":"2024-09-18T01:05:19.705027+0000","objects_scrubbed":1,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:33.647939+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.000630143,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6147,"num_read_kb":49429,"num_write":295,"num_write_kb":13363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":288,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:18.553307+0000","last_change":"2024-09-18T01:05:18.553356+0000","last_active":"2024-09-18T01:05:18.553307+0000","last_peered":"2024-09-18T01:05:18.553307+0000","last_clean":"2024-09-18T01:05:18.553307+0000","last_became_active":"2024-09-18T01:02:32.375219+0000","last_became_peered":"2024-09-18T01:02:32.375219+0000","last_unstale":"2024-09-18T01:05:18.553307+0000","last_undegraded":"2024-09-18T01:05:18.553307+0000","last_fullsized":"2024-09-18T01:05:18.553307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:05:18.553168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:05:18.553168+0000","last_clean_scrub_stamp":"2024-09-18T01:05:18.553168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:55:31.715088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047321700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"148'381","reported_seq":4614,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:18.227052+0000","last_change":"2024-09-18T01:05:18.227117+0000","last_active":"2024-09-18T01:05:18.227052+0000","last_peered":"2024-09-18T01:05:18.227052+0000","last_clean":"2024-09-18T01:05:18.227052+0000","last_became_active":"2024-09-18T01:02:41.454246+0000","last_became_peered":"2024-09-18T01:02:41.454246+0000","last_unstale":"2024-09-18T01:05:18.227052+0000","last_undegraded":"2024-09-18T01:05:18.227052+0000","last_fullsized":"2024-09-18T01:05:18.227052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'381","last_scrub_stamp":"2024-09-18T01:05:18.226928+0000","last_deep_scrub":"148'381","last_deep_scrub_stamp":"2024-09-18T01:05:18.226928+0000","last_clean_scrub_stamp":"2024-09-18T01:05:18.226928+0000","objects_scrubbed":6,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:55:40.758396+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00095897699999999996,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4888,"num_read_kb":30588,"num_write":317,"num_write_kb":9936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":289,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:21.351682+0000","last_change":"2024-09-18T01:05:21.351727+0000","last_active":"2024-09-18T01:05:21.351682+0000","last_peered":"2024-09-18T01:05:21.351682+0000","last_clean":"2024-09-18T01:05:21.351682+0000","last_became_active":"2024-09-18T01:02:32.373864+0000","last_became_peered":"2024-09-18T01:02:32.373864+0000","last_unstale":"2024-09-18T01:05:21.351682+0000","last_undegraded":"2024-09-18T01:05:21.351682+0000","last_fullsized":"2024-09-18T01:05:21.351682+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T01:05:21.351587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T01:05:21.351587+0000","last_clean_scrub_stamp":"2024-09-18T01:05:21.351587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:40.647594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040643800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"148'493","reported_seq":7265,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:24.613445+0000","last_change":"2024-09-18T01:05:24.613490+0000","last_active":"2024-09-18T01:05:24.613445+0000","last_peered":"2024-09-18T01:05:24.613445+0000","last_clean":"2024-09-18T01:05:24.613445+0000","last_became_active":"2024-09-18T01:04:57.303668+0000","last_became_peered":"2024-09-18T01:04:57.303668+0000","last_unstale":"2024-09-18T01:05:24.613445+0000","last_undegraded":"2024-09-18T01:05:24.613445+0000","last_fullsized":"2024-09-18T01:05:24.613445+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'493","last_scrub_stamp":"2024-09-18T01:05:24.613348+0000","last_deep_scrub":"148'493","last_deep_scrub_stamp":"2024-09-18T01:05:24.613348+0000","last_clean_scrub_stamp":"2024-09-18T01:05:24.613348+0000","objects_scrubbed":6,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:30.282028+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00089200800000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7474,"num_read_kb":41986,"num_write":440,"num_write_kb":8938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":16,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"148'2","reported_seq":54,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:19.365953+0000","last_change":"2024-09-18T01:05:19.365987+0000","last_active":"2024-09-18T01:05:19.365953+0000","last_peered":"2024-09-18T01:05:19.365953+0000","last_clean":"2024-09-18T01:05:19.365953+0000","last_became_active":"2024-09-18T01:04:53.298761+0000","last_became_peered":"2024-09-18T01:04:53.298761+0000","last_unstale":"2024-09-18T01:05:19.365953+0000","last_undegraded":"2024-09-18T01:05:19.365953+0000","last_fullsized":"2024-09-18T01:05:19.365953+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'2","last_scrub_stamp":"2024-09-18T01:05:19.365843+0000","last_deep_scrub":"148'2","last_deep_scrub_stamp":"2024-09-18T01:05:19.365843+0000","last_clean_scrub_stamp":"2024-09-18T01:05:19.365843+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:02.549056+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"148'119","reported_seq":443,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:19.315750+0000","last_change":"2024-09-18T01:05:19.315793+0000","last_active":"2024-09-18T01:05:19.315750+0000","last_peered":"2024-09-18T01:05:19.315750+0000","last_clean":"2024-09-18T01:05:19.315750+0000","last_became_active":"2024-09-18T01:02:26.301303+0000","last_became_peered":"2024-09-18T01:02:26.301303+0000","last_unstale":"2024-09-18T01:05:19.315750+0000","last_undegraded":"2024-09-18T01:05:19.315750+0000","last_fullsized":"2024-09-18T01:05:19.315750+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'119","last_scrub_stamp":"2024-09-18T01:05:19.315650+0000","last_deep_scrub":"148'119","last_deep_scrub_stamp":"2024-09-18T01:05:19.315650+0000","last_clean_scrub_stamp":"2024-09-18T01:05:19.315650+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:45.368545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"148'691","reported_seq":8417,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:19.599366+0000","last_change":"2024-09-18T01:05:19.599424+0000","last_active":"2024-09-18T01:05:19.599366+0000","last_peered":"2024-09-18T01:05:19.599366+0000","last_clean":"2024-09-18T01:05:19.599366+0000","last_became_active":"2024-09-18T01:02:41.456637+0000","last_became_peered":"2024-09-18T01:02:41.456637+0000","last_unstale":"2024-09-18T01:05:19.599366+0000","last_undegraded":"2024-09-18T01:05:19.599366+0000","last_fullsized":"2024-09-18T01:05:19.599366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'691","last_scrub_stamp":"2024-09-18T01:05:19.599280+0000","last_deep_scrub":"148'691","last_deep_scrub_stamp":"2024-09-18T01:05:19.599280+0000","last_clean_scrub_stamp":"2024-09-18T01:05:19.599280+0000","objects_scrubbed":7,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:12:26.175796+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00057224499999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8727,"num_read_kb":61019,"num_write":872,"num_write_kb":15570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"148'447","reported_seq":5128,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:22.664109+0000","last_change":"2024-09-18T01:05:22.664189+0000","last_active":"2024-09-18T01:05:22.664109+0000","last_peered":"2024-09-18T01:05:22.664109+0000","last_clean":"2024-09-18T01:05:22.664109+0000","last_became_active":"2024-09-18T01:04:28.341883+0000","last_became_peered":"2024-09-18T01:04:28.341883+0000","last_unstale":"2024-09-18T01:05:22.664109+0000","last_undegraded":"2024-09-18T01:05:22.664109+0000","last_fullsized":"2024-09-18T01:05:22.664109+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'447","last_scrub_stamp":"2024-09-18T01:05:22.664005+0000","last_deep_scrub":"148'447","last_deep_scrub_stamp":"2024-09-18T01:05:22.664005+0000","last_clean_scrub_stamp":"2024-09-18T01:05:22.664005+0000","objects_scrubbed":3,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:44:24.445953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001577216,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":4910,"num_read_kb":30805,"num_write":520,"num_write_kb":10694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":103966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"148'770","reported_seq":5477,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:25.274957+0000","last_change":"2024-09-18T01:05:25.274993+0000","last_active":"2024-09-18T01:05:25.274957+0000","last_peered":"2024-09-18T01:05:25.274957+0000","last_clean":"2024-09-18T01:05:25.274957+0000","last_became_active":"2024-09-18T01:02:41.454679+0000","last_became_peered":"2024-09-18T01:02:41.454679+0000","last_unstale":"2024-09-18T01:05:25.274957+0000","last_undegraded":"2024-09-18T01:05:25.274957+0000","last_fullsized":"2024-09-18T01:05:25.274957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'770","last_scrub_stamp":"2024-09-18T01:05:25.274854+0000","last_deep_scrub":"148'770","last_deep_scrub_stamp":"2024-09-18T01:05:25.274854+0000","last_clean_scrub_stamp":"2024-09-18T01:05:25.274854+0000","objects_scrubbed":4,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:02.331510+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0015997139999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":6721,"num_read_kb":22848,"num_write":932,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"148'518","reported_seq":6992,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:21.670892+0000","last_change":"2024-09-18T01:05:21.670937+0000","last_active":"2024-09-18T01:05:21.670892+0000","last_peered":"2024-09-18T01:05:21.670892+0000","last_clean":"2024-09-18T01:05:21.670892+0000","last_became_active":"2024-09-18T01:02:41.455924+0000","last_became_peered":"2024-09-18T01:02:41.455924+0000","last_unstale":"2024-09-18T01:05:21.670892+0000","last_undegraded":"2024-09-18T01:05:21.670892+0000","last_fullsized":"2024-09-18T01:05:21.670892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'518","last_scrub_stamp":"2024-09-18T01:05:21.670792+0000","last_deep_scrub":"148'518","last_deep_scrub_stamp":"2024-09-18T01:05:21.670792+0000","last_clean_scrub_stamp":"2024-09-18T01:05:21.670792+0000","objects_scrubbed":7,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:27:54.369975+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00065765300000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8425,"num_read_kb":38485,"num_write":423,"num_write_kb":6922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"148'376","reported_seq":6095,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:23.376174+0000","last_change":"2024-09-18T01:05:23.376208+0000","last_active":"2024-09-18T01:05:23.376174+0000","last_peered":"2024-09-18T01:05:23.376174+0000","last_clean":"2024-09-18T01:05:23.376174+0000","last_became_active":"2024-09-18T01:04:56.304292+0000","last_became_peered":"2024-09-18T01:04:56.304292+0000","last_unstale":"2024-09-18T01:05:23.376174+0000","last_undegraded":"2024-09-18T01:05:23.376174+0000","last_fullsized":"2024-09-18T01:05:23.376174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'376","last_scrub_stamp":"2024-09-18T01:05:23.376089+0000","last_deep_scrub":"148'376","last_deep_scrub_stamp":"2024-09-18T01:05:23.376089+0000","last_clean_scrub_stamp":"2024-09-18T01:05:23.376089+0000","objects_scrubbed":5,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:59:52.677277+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00095466699999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6219,"num_read_kb":29077,"num_write":316,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"148'536","reported_seq":6232,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:24.279282+0000","last_change":"2024-09-18T01:05:24.279319+0000","last_active":"2024-09-18T01:05:24.279282+0000","last_peered":"2024-09-18T01:05:24.279282+0000","last_clean":"2024-09-18T01:05:24.279282+0000","last_became_active":"2024-09-18T01:02:41.454059+0000","last_became_peered":"2024-09-18T01:02:41.454059+0000","last_unstale":"2024-09-18T01:05:24.279282+0000","last_undegraded":"2024-09-18T01:05:24.279282+0000","last_fullsized":"2024-09-18T01:05:24.279282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'536","last_scrub_stamp":"2024-09-18T01:05:24.279192+0000","last_deep_scrub":"148'536","last_deep_scrub_stamp":"2024-09-18T01:05:24.279192+0000","last_clean_scrub_stamp":"2024-09-18T01:05:24.279192+0000","objects_scrubbed":6,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:23.213915+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013470540000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6693,"num_read_kb":28822,"num_write":535,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"148'406","reported_seq":6905,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:26.458164+0000","last_change":"2024-09-18T01:05:26.458220+0000","last_active":"2024-09-18T01:05:26.458164+0000","last_peered":"2024-09-18T01:05:26.458164+0000","last_clean":"2024-09-18T01:05:26.458164+0000","last_became_active":"2024-09-18T01:04:56.306823+0000","last_became_peered":"2024-09-18T01:04:56.306823+0000","last_unstale":"2024-09-18T01:05:26.458164+0000","last_undegraded":"2024-09-18T01:05:26.458164+0000","last_fullsized":"2024-09-18T01:05:26.458164+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'406","last_scrub_stamp":"2024-09-18T01:05:26.458053+0000","last_deep_scrub":"148'406","last_deep_scrub_stamp":"2024-09-18T01:05:26.458053+0000","last_clean_scrub_stamp":"2024-09-18T01:05:26.458053+0000","objects_scrubbed":7,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:58:57.685267+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00093209,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6792,"num_read_kb":36900,"num_write":342,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":16384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"148'618","reported_seq":13239,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:22.350762+0000","last_change":"2024-09-18T01:05:22.350807+0000","last_active":"2024-09-18T01:05:22.350762+0000","last_peered":"2024-09-18T01:05:22.350762+0000","last_clean":"2024-09-18T01:05:22.350762+0000","last_became_active":"2024-09-18T01:04:56.303159+0000","last_became_peered":"2024-09-18T01:04:56.303159+0000","last_unstale":"2024-09-18T01:05:22.350762+0000","last_undegraded":"2024-09-18T01:05:22.350762+0000","last_fullsized":"2024-09-18T01:05:22.350762+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'618","last_scrub_stamp":"2024-09-18T01:05:22.350668+0000","last_deep_scrub":"148'618","last_deep_scrub_stamp":"2024-09-18T01:05:22.350668+0000","last_clean_scrub_stamp":"2024-09-18T01:05:22.350668+0000","objects_scrubbed":4,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:49:19.035478+0000","scrub_duration":3,"objects_trimmed":4,"snaptrim_duration":0.0013884469999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11110,"num_read_kb":26027,"num_write":450,"num_write_kb":7509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"148'444","reported_seq":4655,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:24.424748+0000","last_change":"2024-09-18T01:05:24.424786+0000","last_active":"2024-09-18T01:05:24.424748+0000","last_peered":"2024-09-18T01:05:24.424748+0000","last_clean":"2024-09-18T01:05:24.424748+0000","last_became_active":"2024-09-18T01:04:56.303327+0000","last_became_peered":"2024-09-18T01:04:56.303327+0000","last_unstale":"2024-09-18T01:05:24.424748+0000","last_undegraded":"2024-09-18T01:05:24.424748+0000","last_fullsized":"2024-09-18T01:05:24.424748+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'444","last_scrub_stamp":"2024-09-18T01:05:24.424661+0000","last_deep_scrub":"148'444","last_deep_scrub_stamp":"2024-09-18T01:05:24.424661+0000","last_clean_scrub_stamp":"2024-09-18T01:05:24.424661+0000","objects_scrubbed":5,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:26.602755+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000651806,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":5014,"num_read_kb":25886,"num_write":432,"num_write_kb":7619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2224957,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"148'413","reported_seq":4547,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:23.318709+0000","last_change":"2024-09-18T01:05:23.318766+0000","last_active":"2024-09-18T01:05:23.318709+0000","last_peered":"2024-09-18T01:05:23.318709+0000","last_clean":"2024-09-18T01:05:23.318709+0000","last_became_active":"2024-09-18T01:02:41.453867+0000","last_became_peered":"2024-09-18T01:02:41.453867+0000","last_unstale":"2024-09-18T01:05:23.318709+0000","last_undegraded":"2024-09-18T01:05:23.318709+0000","last_fullsized":"2024-09-18T01:05:23.318709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'413","last_scrub_stamp":"2024-09-18T01:05:23.318619+0000","last_deep_scrub":"148'413","last_deep_scrub_stamp":"2024-09-18T01:05:23.318619+0000","last_clean_scrub_stamp":"2024-09-18T01:05:23.318619+0000","objects_scrubbed":5,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:56:04.730942+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00092681000000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4281,"num_read_kb":19663,"num_write":387,"num_write_kb":3478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"148'979","reported_seq":10516,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:17.186997+0000","last_change":"2024-09-18T01:05:17.187027+0000","last_active":"2024-09-18T01:05:17.186997+0000","last_peered":"2024-09-18T01:05:17.186997+0000","last_clean":"2024-09-18T01:05:17.186997+0000","last_became_active":"2024-09-18T01:02:41.451701+0000","last_became_peered":"2024-09-18T01:02:41.451701+0000","last_unstale":"2024-09-18T01:05:17.186997+0000","last_undegraded":"2024-09-18T01:05:17.186997+0000","last_fullsized":"2024-09-18T01:05:17.186997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'979","last_scrub_stamp":"2024-09-18T01:05:17.186914+0000","last_deep_scrub":"148'979","last_deep_scrub_stamp":"2024-09-18T01:05:17.186914+0000","last_clean_scrub_stamp":"2024-09-18T01:05:17.186914+0000","objects_scrubbed":3,"log_size":979,"log_dups_size":0,"ondisk_log_size":979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:16:34.178800+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00071876100000000005,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":12794,"num_read_kb":52335,"num_write":1236,"num_write_kb":15856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"148'320","reported_seq":4132,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-18T01:05:28.214892+0000","last_change":"2024-09-18T01:05:28.214968+0000","last_active":"2024-09-18T01:05:28.214892+0000","last_peered":"2024-09-18T01:05:28.214892+0000","last_clean":"2024-09-18T01:05:28.214892+0000","last_became_active":"2024-09-18T01:04:56.302746+0000","last_became_peered":"2024-09-18T01:04:56.302746+0000","last_unstale":"2024-09-18T01:05:28.214892+0000","last_undegraded":"2024-09-18T01:05:28.214892+0000","last_fullsized":"2024-09-18T01:05:28.214892+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'320","last_scrub_stamp":"2024-09-18T01:05:28.214791+0000","last_deep_scrub":"148'320","last_deep_scrub_stamp":"2024-09-18T01:05:28.214791+0000","last_clean_scrub_stamp":"2024-09-18T01:05:28.214791+0000","objects_scrubbed":7,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:06:48.480421+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0016537360000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3257,"num_read_kb":13085,"num_write":295,"num_write_kb":7883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":179,"num_object_clones":0,"num_object_copies":358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":175,"num_read":265609,"num_read_kb":1200484,"num_write":22927,"num_write_kb":286409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2357,"num_bytes_recovered":32233456,"num_keys_recovered":85,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20780,"ondisk_log_size":20780,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17100,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640297,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41236,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330604,"statfs":{"total":96636764160,"available":96594538496,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640299,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46804,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325036,"statfs":{"total":96636764160,"available":96588836864,"internally_reserved":0,"allocated":1773568,"data_stored":730466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640298,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51016,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320824,"statfs":{"total":96636764160,"available":96584523776,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640297,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77328,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294512,"statfs":{"total":96636764160,"available":96557580288,"internally_reserved":0,"allocated":1769472,"data_stored":730447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640299,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64856,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306984,"statfs":{"total":96636764160,"available":96570351616,"internally_reserved":0,"allocated":1777664,"data_stored":730485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640298,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46856,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324984,"statfs":{"total":96636764160,"available":96588783616,"internally_reserved":0,"allocated":1826816,"data_stored":1320815,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8550,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T01:05:39.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T01:05:40.260 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T01:05:40.260 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T01:05:40.261 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T01:05:40.261 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.300 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T01:05:40.300 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T01:05:40.300 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.371 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T01:05:40.372 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T01:05:40.372 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.449 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T01:05:40.450 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T01:05:40.450 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.495 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T01:05:40.496 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T01:05:40.496 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.549 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T01:05:40.549 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T01:05:40.550 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.590 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T01:05:40.590 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T01:05:40.590 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T01:05:40.591 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.660 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T01:05:40.661 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T01:05:40.661 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T01:05:40.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.673 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T01:05:40.673 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T01:05:40.673 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.724 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T01:05:40.725 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T01:05:40.725 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T01:05:40.737 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T01:05:40.737 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T01:05:40.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-18T01:05:40.804 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:05:40.804 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T01:05:40.986 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:05:40.987 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T01:05:41.052 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:05:41.052 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T01:05:41.110 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:05:41.111 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T01:05:41.318 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:05:41.318 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T01:05:41.416 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:05:41.417 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T01:05:41.480 INFO:tasks.ceph:Archiving mon data... 2024-09-18T01:05:41.481 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/data/mon.a.tgz 2024-09-18T01:05:41.481 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-18T01:05:41.497 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.ohAC5Js4Hu 2024-09-18T01:05:41.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ohAC5Js4Hu 2024-09-18T01:05:41.644 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.ohAC5Js4Hu 2024-09-18T01:05:41.729 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.ohAC5Js4Hu is 300KB 2024-09-18T01:05:41.825 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.ohAC5Js4Hu 2024-09-18T01:05:41.841 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/data/mon.c.tgz 2024-09-18T01:05:41.841 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-18T01:05:41.897 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.P9fcIian07 2024-09-18T01:05:41.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.P9fcIian07 2024-09-18T01:05:42.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.P9fcIian07 2024-09-18T01:05:42.133 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.P9fcIian07 is 404KB 2024-09-18T01:05:42.231 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.P9fcIian07 2024-09-18T01:05:42.246 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/data/mon.b.tgz 2024-09-18T01:05:42.246 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-18T01:05:42.265 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.ltYTqgBPVf 2024-09-18T01:05:42.265 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ltYTqgBPVf 2024-09-18T01:05:42.429 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.ltYTqgBPVf 2024-09-18T01:05:42.513 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.ltYTqgBPVf is 404KB 2024-09-18T01:05:42.609 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.ltYTqgBPVf 2024-09-18T01:05:42.624 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T01:05:42.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T01:05:42.627 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T01:05:42.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T01:05:42.716 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T01:05:42.743 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T01:05:42.761 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-18T01:05:42.762 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T01:05:42.763 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi031/crash 2024-09-18T01:05:42.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T01:05:42.790 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi098/crash 2024-09-18T01:05:42.791 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T01:05:42.816 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi164/crash 2024-09-18T01:05:42.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T01:05:42.948 INFO:tasks.ceph:Compressing logs... 2024-09-18T01:05:42.949 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T01:05:42.951 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T01:05:42.953 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T01:05:42.975 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T01:05:42.983 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31419.log 2024-09-18T01:05:42.983 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31595.log 2024-09-18T01:05:42.983 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T01:05:42.983 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.31419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31419.log.gz 2024-09-18T01:05:42.983 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.31595.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T01:05:42.983 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31595.log.gz 2024-09-18T01:05:42.983 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31475.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31635.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.31475.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31475.log.gz -5 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr: --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.31635.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31635.log.gz 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31555.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.31555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31515.log 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31555.log.gz 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T01:05:42.984 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T01:05:42.985 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.31515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31515.log.gz 2024-09-18T01:05:43.049 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.3.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T01:05:43.069 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28746.log 2024-09-18T01:05:43.069 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28290.log 2024-09-18T01:05:43.069 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28965.log 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28746.log.gz 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:gzip/var/log/ceph/ceph-client.admin.28290.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28850.log 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28290.log.gz 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28405.log 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28850.log: 95.9% -- replaced with /var/log/ceph/ceph-client.admin.28965.log.gz 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27529.log 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28850.log.gz 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27669.log 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.28405.log.gz 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28719.log 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr: 35.4% -- replaced with /var/log/ceph/ceph-client.admin.27529.log.gz 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27388.log 2024-09-18T01:05:43.070 INFO:teuthology.orchestra.run.smithi164.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.27669.log.gz 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27459.log 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.28719.log.gz 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27450.log 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.27388.log.gz 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29419.log 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27459.log: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.27459.log.gz 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29139.log 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27450.log.gz 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27319.log 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.29419.log.gz 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29105.log 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.29139.log.gz 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28710.log 2024-09-18T01:05:43.071 INFO:teuthology.orchestra.run.smithi164.stderr: 51.8% -- replaced with /var/log/ceph/ceph-client.admin.27319.log.gz 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28055.log 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.29105.log.gz 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28710.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28710.log.gz 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28265.log 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27184.log 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.28055.log.gz 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.27184.log: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.28265.log.gz 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29270.log 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.27184.log.gz 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28780.log 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-18T01:05:43.072 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27766.log 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29270.log.gz 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28536.log 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28780.log.gz 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29455.log 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27766.log.gz 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28999.log 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28536.log.gz 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28859.log 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.29455.log.gz 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27800.log 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28999.log.gz 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27217.log 2024-09-18T01:05:43.073 INFO:teuthology.orchestra.run.smithi164.stderr: 51.1% -- replaced with /var/log/ceph/ceph-client.admin.28859.log.gz 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29315.log 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27800.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27800.log.gz 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28256.log 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27217.log.gz 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27739.log 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.29315.log.gz 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28159.log 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28256.log.gz 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28335.log 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.27739.log.gz 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28186.log 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.28159.log.gz 2024-09-18T01:05:43.074 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29060.log 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.28335.log.gz 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27635.log 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28186.log.gz 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28615.log 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29060.log.gz 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28649.log 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27425.log 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.admin.28615.log.gz 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28649.log: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.27635.log.gz 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29349.log 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.28649.log.gz 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29096.log 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.27425.log.gz 2024-09-18T01:05:43.075 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27915.log 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.29349.log.gz 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29166.log 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29096.log.gz 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28755.log 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.27915.log.gz 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27354.log 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29166.log.gz 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28895.log 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.28755.log.gz 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28789.log 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28895.log: 96.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.27354.log.gz -5 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr: --verbose -- /var/log/ceph/ceph-client.admin.28920.log 2024-09-18T01:05:43.076 INFO:teuthology.orchestra.run.smithi164.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.28895.log.gz 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27696.log 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28789.log: 54.8% -- replaced with /var/log/ceph/ceph-client.admin.28789.log.gz 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29446.log 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.28920.log.gz 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29340.log 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27696.log.gz 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27809.log 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29446.log.gz 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28685.log 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29340.log.gz 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28475.log 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.27809.log.gz 2024-09-18T01:05:43.077 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28466.log 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.28685.log.gz 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28825.log 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.28475.log.gz 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28195.log 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28466.log.gz 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29410.log 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.28825.log.gz 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27985.log 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29410.log: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.28195.log.gz 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28360.log 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29410.log.gz 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28886.log 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.27985.log.gz 2024-09-18T01:05:43.078 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28299.log 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28360.log.gz 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29306.log 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28886.log.gz 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28606.log 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.28299.log.gz 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27845.log 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29306.log.gz 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28116.log 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28606.log.gz 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27416.log 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.27845.log.gz 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29200.log 2024-09-18T01:05:43.079 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28116.log.gz 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27556.log 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27416.log.gz 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28990.log 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29200.log.gz 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28816.log 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27556.log.gz 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27949.log 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28990.log.gz 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27565.log 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.28816.log.gz 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29130.log 2024-09-18T01:05:43.080 INFO:teuthology.orchestra.run.smithi164.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.27949.log.gz 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28150.log 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.27565.log.gz 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28089.log 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29130.log.gz 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28956.log 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28150.log.gz 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28369.log 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28089.log.gz 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28080.log 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28956.log.gz 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28439.log 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.28369.log.gz 2024-09-18T01:05:43.081 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29236.log 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28080.log.gz 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29245.log 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28439.log: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.28439.log.gz 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27879.log 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29236.log.gz 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27705.log 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr: 54.8% -- replaced with /var/log/ceph/ceph-client.admin.29245.log.gz 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27345.log 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.27879.log.gz 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27226.log 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.27705.log.gz 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28570.log 2024-09-18T01:05:43.082 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27345.log.gz 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28046.log 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr: 56.3% -- replaced with /var/log/ceph/ceph-client.admin.27226.log.gz 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27379.log 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28570.log.gz 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28019.log 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28046.log.gz 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27976.log 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27379.log.gz 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29376.log 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.28019.log.gz 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29026.log 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27976.log.gz 2024-09-18T01:05:43.083 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28125.log 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29376.log.gz 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27660.log 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29026.log.gz 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28509.log 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.28125.log.gz 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27495.log 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27660.log.gz 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27906.log 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.28509.log.gz 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28326.log 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr: 54.9% -- replaced with /var/log/ceph/ceph-client.admin.27495.log.gz 2024-09-18T01:05:43.084 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29209.log 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27906.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27906.log.gz 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29279.log 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28326.log.gz 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27590.log 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.29209.log.gz 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29175.log 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr: 35.3% -- replaced with /var/log/ceph/ceph-client.admin.29279.log.gz 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28640.log 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27590.log.gz 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27626.log 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28545.log 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28640.log.gz 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.29175.log.gz 2024-09-18T01:05:43.085 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27486.log 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27626.log.gz 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28220.log 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28545.log.gz 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28010.log 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27486.log.gz 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27310.log 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28220.log.gz 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29385.log 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28010.log.gz 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27870.log 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27310.log.gz 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27599.log 2024-09-18T01:05:43.086 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28430.log 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.29385.log.gz 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27870.log.gz 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27940.log 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.27599.log.gz 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29069.log 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28430.log.gz 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27730.log 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27940.log.gz 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28396.log 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.29069.log.gz 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29035.log 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27730.log.gz 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27520.log 2024-09-18T01:05:43.087 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28396.log.gz 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28676.log 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28229.log 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.29035.log.gz 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27520.log.gz 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28500.log 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28676.log.gz 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28579.log 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.28229.log.gz 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27836.log 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28500.log.gz 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27775.log 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.28579.log.gz 2024-09-18T01:05:43.088 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27836.log: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27836.log.gz 2024-09-18T01:05:43.089 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27775.log: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.27775.log.gz 2024-09-18T01:05:43.089 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-18T01:05:43.089 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m0.052s 2024-09-18T01:05:43.089 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m0.054s 2024-09-18T01:05:43.089 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.072s 2024-09-18T01:05:43.129 INFO:teuthology.orchestra.run.smithi098.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48438.log 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48438.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48438.log.gz 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41214.log 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42464.log 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41214.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41214.log.gz 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38099.log 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47448.log 2024-09-18T01:05:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42464.log.gz 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35953.log 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38099.log.gz 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34979.log 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47448.log.gz 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35953.log.gzgzip 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52433.log 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34979.log.gz 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51546.log 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52433.log.gz 2024-09-18T01:05:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38467.log 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36736.log 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51546.log.gz 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38467.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.47373.log -- replaced with /var/log/ceph/ceph-client.admin.38467.log.gz 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40486.log 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36736.log.gz 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47373.log.gz 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45899.log 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40486.log.gz 2024-09-18T01:05:43.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36777.log 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34544.log 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45899.log.gz 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36777.log.gz 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34544.log.gz 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39832.log 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37136.log 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53567.log 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39832.log.gz 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2024-09-18T01:05:43.155 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37136.log.gz 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53567.log.gz 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52655.log 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39536.log 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34982.log 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52655.log.gz 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39536.log.gz 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51229.log 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34982.log.gz 2024-09-18T01:05:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36377.log 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42239.log 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51229.log.gz 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43052.log 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36377.log.gz 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52475.log 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42239.log.gz 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43052.log.gz 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42763.log 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52475.log.gz 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41488.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T01:05:43.157 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42763.log.gz 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45316.log 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50831.log 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45316.log.gz 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50831.log.gz 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52849.log 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43871.log 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43658.log 2024-09-18T01:05:43.158 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52849.log.gz 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53861.log 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43871.log.gz 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43658.log.gz 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35765.log 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50954.log 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53861.log.gz 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35765.log.gz 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43254.log 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44127.log 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50954.log.gz 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43254.log.gz 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38374.log 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2024-09-18T01:05:43.159 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44127.log.gz 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38374.log.gz 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50675.log 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40384.log 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50675.log.gz 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36399.log 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41982.log 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40384.log.gz 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36399.log.gz 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47766.log 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53945.log 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41982.log.gz 2024-09-18T01:05:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47766.log.gz 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43611.log 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47293.log 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53945.log.gz 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43611.log.gz 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35849.log 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47293.log: -- /var/log/ceph/ceph-client.admin.43708.log 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47293.log.gz 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35849.log.gz 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34978.log 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49709.log 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43708.log.gz 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38744.log 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34978.log.gz 2024-09-18T01:05:43.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49709.log.gz 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38744.log.gz 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46977.log 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46977.log.gz 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48357.log 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gz 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48357.log.gz 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45472.log 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53063.log 2024-09-18T01:05:43.162 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42557.log 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45472.log.gz 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41448.log 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53063.log.gz 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50314.log 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42557.log.gz 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41448.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.41448.log.gz -- 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.49656.log 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34983.log 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50314.log.gz 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46570.log 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50110.log 2024-09-18T01:05:43.163 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34983.log.gz 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46570.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46570.log.gz -- 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.46215.log 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50613.log 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50110.log.gz 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46215.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.46215.log.gz --verbose 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph.audit.log 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52097.log 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50613.log.gz 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35407.log 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52097.log.gz 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-09-18T01:05:43.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35407.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41000.log 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38394.log 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37176.log 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41000.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42082.log 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38394.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37866.log 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37176.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42082.log.gz 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49873.log 2024-09-18T01:05:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53008.log 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37866.log.gz 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49873.log.gz 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39719.log 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53008.log.gz 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44744.log 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41838.log 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39719.log.gz 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44744.log.gzgzip 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40878.log 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36075.log 2024-09-18T01:05:43.166 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41838.log.gz 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51630.log 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40878.log.gz 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53399.log 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36075.log.gz 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51630.log: -- /var/log/ceph/ceph-client.admin.50014.log 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51630.log.gz 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50469.log 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53399.log.gz 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53189.log 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50014.log.gz 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40466.log 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50469.log.gz 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36831.log 2024-09-18T01:05:43.167 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53189.log.gz 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51401.log 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40466.log.gz 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35639.log 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36831.log.gz 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53525.log 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52658.log 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35639.log.gz 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53525.log.gz 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39144.log 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52658.log.gz 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49381.log 2024-09-18T01:05:43.168 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gz 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46195.log 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39144.log.gz 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49381.log.gz 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39101.log 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46195.log.gz 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44426.log 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40546.log/var/log/ceph/ceph-client.admin.39101.log: 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39101.log.gz 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44426.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.44426.log.gz --verbose 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.47120.log 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39038.log 2024-09-18T01:05:43.169 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40546.log.gz 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47120.log.gz 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39038.log.gz 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34625.log 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48953.log 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35807.log 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34625.log.gz 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35387.log 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48953.log.gz 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53315.log 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35807.log.gz 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52657.log 2024-09-18T01:05:43.170 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35387.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52908.log 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53315.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35503.log 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52657.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47657.log 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52908.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53483.log 0.0% 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35503.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52909.log 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47657.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43307.log 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52909.log.gz 2024-09-18T01:05:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50511.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49421.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43307.log.gz 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42979.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50511.log.gz 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38806.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49421.log.gz 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46682.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42979.log.gz 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48082.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38806.log.gz 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52181.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46682.log.gz 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47592.log 2024-09-18T01:05:43.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48082.log.gz 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52848.log 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52181.log.gz 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47592.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47592.log.gz -- 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.50735.log 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35507.log 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52848.log.gz 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43750.log 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50735.log.gz 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35507.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.39248.log 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35507.log.gz 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51971.log 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43750.log.gz 2024-09-18T01:05:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39248.log.gz 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37033.log 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51971.log.gz 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44220.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz -5 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48124.log 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46411.log 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37033.log.gz 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48124.log.gz 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52654.log 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46845.log 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46411.log.gz 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52654.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52654.log.gz 2024-09-18T01:05:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35215.log 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46845.log.gz 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51734.log 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35212.log 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35215.log.gz 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52656.log 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51734.log.gz 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35212.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35212.log.gz 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38631.log 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52656.log.gz 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41763.log 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38631.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38631.log.gz 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37229.log 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41763.log.gz 2024-09-18T01:05:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42704.log 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44342.log 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37229.log.gz 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53819.log 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42704.log.gz 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.44342.log: -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27215.log 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44342.log.gz 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53819.log.gz 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.tmp-client.admin.27215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34799.log 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27215.log.gz 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46488.log 2024-09-18T01:05:43.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34799.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34799.log.gz 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42620.log 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46488.log.gz 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31900.log 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.42620.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42620.log.gz 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35457.log 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31900.log.gz 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39186.log 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35457.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35457.log.gz 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34681.log 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39186.log.gz 2024-09-18T01:05:43.177 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48535.log 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34681.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34681.log.gz 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41153.log 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48535.log.gz 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36315.log 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41153.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41153.log.gz 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53147.log 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36315.log.gz 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35170.log 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53147.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53147.log.gz 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37824.log 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35170.log.gz 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51650.log 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37824.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37824.log.gz 2024-09-18T01:05:43.178 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31826.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51650.log.gz 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39852.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.31826.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31826.log.gz 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52659.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39852.log.gz 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51924.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52659.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52659.log.gz 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41723.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51924.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.49148.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51924.log.gz 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41723.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41723.log.gz 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49148.log: -- /var/log/ceph/ceph-client.admin.43892.log 2024-09-18T01:05:43.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49148.log.gz 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53273.log 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43892.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50130.log 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47869.log 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53273.log: /var/log/ceph/ceph-client.admin.50130.log: 0.0% 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53273.log.gz 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50130.log.gz 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44384.log 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47869.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47869.log.gz 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52391.log 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44384.log.gz 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52801.log 2024-09-18T01:05:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52391.log.gz 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34981.log 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52801.log.gz 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35528.log 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34981.log.gz 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34841.log 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35528.log.gz 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54027.log 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34841.log.gz 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54027.log.gz 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39039.log 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-09-18T01:05:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35891.log 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39039.log.gz 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52223.log 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35891.log.gz 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47510.log 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52223.log.gz 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47510.log.gz 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35681.log 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-09-18T01:05:43.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35681.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35681.log.gz 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53903.log 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46042.log 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53903.log: /var/log/ceph/ceph-client.admin.44487.log 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53903.log.gz 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46042.log.gz 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44487.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44487.log.gz 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52349.log 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43389.log 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52349.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52349.log.gz 2024-09-18T01:05:43.183 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35174.log 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43389.log.gz 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52307.log 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35174.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35174.log.gz 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45767.log 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52307.log.gz 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49086.log 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45767.log.gz 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52265.log 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49086.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49086.log.gz 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39504.log 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52265.log.gz 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42177.log 2024-09-18T01:05:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39504.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39504.log.gz 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37013.log 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42177.log.gz 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51047.log 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37013.log.gz 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44827.log 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51047.log: -- /var/log/ceph/ceph-client.admin.46314.log 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51047.log.gz/var/log/ceph/ceph-client.admin.44827.log: 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52139.log 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44827.log.gz 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46314.log: /var/log/ceph/ceph-client.admin.51882.log 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46314.log.gz 2024-09-18T01:05:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52139.log.gz 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53231.log 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51882.log: gzip/var/log/ceph/ceph-client.admin.41837.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37762.log 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51882.log.gz 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53231.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53231.log.gz 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39955.log 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37762.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37762.log.gz 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51588.log 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39955.log.gz 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48738.log 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51588.log.gz 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45101.log 2024-09-18T01:05:43.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48738.log.gz 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42662.log 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45101.log.gz 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42662.log.gz 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53777.log 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53777.log.gz 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35235.log 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48397.log 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35235.log.gz 2024-09-18T01:05:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51733.log 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48397.log.gz 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51733.log.gz 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53987.log 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53105.log 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53987.log.gz 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50553.log 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53105.log.gz 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39226.log 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50553.log.gz 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50068.log 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39226.log.gz 2024-09-18T01:05:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52055.log 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50068.log.gz 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31597.log 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52055.log.gz 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.31597.log: /var/log/ceph/ceph-client.admin.34974.log 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr: 28.8% -- replaced with /var/log/ceph/ceph-client.admin.31597.log.gz/var/log/ceph/ceph-client.admin.53441.log: 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44488.log 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46315.log 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34974.log.gzgzip 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44488.log: /var/log/ceph/ceph-client.admin.46315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44488.log.gz 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46315.log.gz 2024-09-18T01:05:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36076.log 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34706.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40597.log 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36076.log.gz 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40149.log 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52013.log 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40597.log.gz 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40190.log 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40149.log.gz 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5/var/log/ceph/ceph-client.admin.52013.log: --verbose -- /var/log/ceph/ceph-client.admin.43409.log 0.0% 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52013.log.gz 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40190.log.gz 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38140.log 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34757.log 2024-09-18T01:05:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43409.log: /var/log/ceph/ceph-client.admin.38140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43409.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38140.log.gz 2024-09-18T01:05:43.191 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-18T01:05:43.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34757.log.gz 2024-09-18T01:05:43.191 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-18T01:05:43.192 INFO:teuthology.orchestra.run.smithi098.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T01:05:43.296 INFO:teuthology.orchestra.run.smithi031.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T01:05:47.710 INFO:teuthology.orchestra.run.smithi031.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T01:05:47.775 INFO:teuthology.orchestra.run.smithi098.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T01:05:47.782 INFO:teuthology.orchestra.run.smithi031.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T01:05:48.160 INFO:teuthology.orchestra.run.smithi031.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T01:05:48.163 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-18T01:05:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m5.198s 2024-09-18T01:05:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m14.627s 2024-09-18T01:05:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.636s 2024-09-18T01:05:48.984 INFO:teuthology.orchestra.run.smithi098.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T01:05:48.986 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-18T01:05:48.986 INFO:teuthology.orchestra.run.smithi098.stderr:real 0m6.020s 2024-09-18T01:05:48.986 INFO:teuthology.orchestra.run.smithi098.stderr:user 0m10.804s 2024-09-18T01:05:48.986 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m0.434s 2024-09-18T01:05:48.987 INFO:tasks.ceph:Archiving logs... 2024-09-18T01:05:48.987 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi031/log 2024-09-18T01:05:48.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T01:05:50.879 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi098/log 2024-09-18T01:05:50.880 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T01:05:52.240 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi164/log 2024-09-18T01:05:52.241 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T01:05:52.350 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T01:05:52.371 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T01:05:52.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T01:05:52.374 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T01:05:52.376 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T01:05:52.477 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T01:05:52.486 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T01:05:52.486 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:05:52.489 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:05:52.491 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T01:05:52.505 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-18T01:05:52.505 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-18T01:05:52.509 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-18T01:05:52.509 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T01:05:52.509 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-18T01:05:52.509 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -264us[ -268us] +/- 45ms 2024-09-18T01:05:52.509 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -1777us[-1781us] +/- 80ms 2024-09-18T01:05:52.509 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +1914us[+1911us] +/- 68ms 2024-09-18T01:05:52.510 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:05:52.511 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T01:05:52.511 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-18T01:05:52.511 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -236us[ -223us] +/- 45ms 2024-09-18T01:05:52.511 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -1778us[-1778us] +/- 80ms 2024-09-18T01:05:52.511 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +1937us[+1950us] +/- 68ms 2024-09-18T01:05:52.511 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:05:52.513 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T01:05:52.513 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-18T01:05:52.513 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 -244us[ -237us] +/- 45ms 2024-09-18T01:05:52.513 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -1772us[-1772us] +/- 80ms 2024-09-18T01:05:52.513 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +1948us[+1948us] +/- 68ms 2024-09-18T01:05:52.514 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T01:05:52.514 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T01:05:52.523 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T01:05:52.523 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T01:05:52.531 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T01:05:52.635 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T01:05:52.709 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T01:05:52.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T01:05:52.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:05:52.808 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T01:05:52.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:05:52.838 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T01:05:52.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:05:52.867 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T01:05:52.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T01:05:52.885 INFO:teuthology.task.internal:Duration was 625.935315 seconds 2024-09-18T01:05:52.885 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T01:05:52.894 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T01:05:52.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T01:05:52.897 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T01:05:52.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T01:05:52.937 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T01:05:52.938 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T01:05:52.949 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T01:05:53.234 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T01:05:53.234 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-18T01:05:53.235 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T01:05:53.257 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-09-18T01:05:53.257 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T01:05:53.300 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-18T01:05:53.301 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T01:05:53.324 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T01:05:53.324 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T01:05:53.327 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T01:05:53.343 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T01:05:53.392 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T01:05:53.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T01:05:53.396 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T01:05:53.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T01:05:53.537 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T01:05:53.550 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T01:05:53.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T01:05:53.576 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T01:05:53.601 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T01:05:53.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T01:05:53.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:05:53.638 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:05:53.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:05:53.660 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-18T01:05:53.664 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-09-18T01:05:53.690 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-18T01:05:53.761 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:05:53.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:05:53.776 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:05:53.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:05:53.791 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T01:05:53.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T01:05:53.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T01:05:53.826 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T01:05:53.827 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi031 2024-09-18T01:05:53.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T01:05:53.862 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi098 2024-09-18T01:05:53.862 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T01:05:53.896 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909691/remote/smithi164 2024-09-18T01:05:53.896 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T01:05:53.932 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T01:05:53.932 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T01:05:53.935 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T01:05:53.937 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T01:05:53.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T01:05:53.994 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T01:05:53.994 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T01:05:54.005 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T01:05:54.005 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T01:05:54.008 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T01:05:54.010 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T01:05:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 01:05 /home/ubuntu/cephtest 2024-09-18T01:05:54.024 INFO:teuthology.orchestra.run.smithi098.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 01:05 /home/ubuntu/cephtest 2024-09-18T01:05:54.037 INFO:teuthology.orchestra.run.smithi164.stdout: 265836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 01:05 /home/ubuntu/cephtest 2024-09-18T01:05:54.039 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T01:05:54.054 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T01:05:54.097 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 625.935314655304 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-18T01:05:54.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T01:05:54.180 INFO:teuthology.run:pass